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FIG.36A 
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FIG.37 
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FIG.38 
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FIG.39 
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FIG.42A 
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MULTIPLE PICTURE COMPOSING the transmission band of the network 204, and applied with 

METHOD AND MULTIPLE PICTURE standardized means such as H.320 (H.261), MPEG1, 2, 4, 

COMPOSING APPARATUS MOTION-JPEG, H.324 (H.263). 

In a case where each television camera is set up in a 

BACKGROUND OF THE INVENTION 5 corresponding video conference room, the transmission 

The present invention generally relates to multiple picture ^ 2 ° 3 ™ d 205 ar f muhiall y C0U P led to a P ar ' of the 

composing methods and multiple picture composing " etW0 * 204 Via * ^U-point .connection umt In this case, 

r . 6 i . • | ] « i.- i • . tne video sigoal distribution is controlled so as to form a 

apparatuses, and more particularly to a mu tiple p.cture ^ tem 

composing method for composing a plurality of pictures and ^ \ ^ j- r i • • 

i * , a * •* 10 FIGS. 2A through 2D are diagrams for explaining the 

transmitting a composed picture and to a multiple picture 30 u . , . . & , r % • * 

. & . . • V , . ! multiple picture composing, and show monitoring screens at 

composing apparatus which employs such a multiple picture rece ^ ng ends of / rem * e monitoring system 6 nG 2A 

composing method. shows a sp iit monitoring screen which includes 4 split 

In video conference systems, remote monitoring systems pictures and is identical to the display screen of the monitor 

and the like, it is known to transmit a plurality of pictures 206 shown in FIG. 1. FIG, 2B shows a split monitoring 

and to make a multiple picture display at a receiving end. In 15 screen which includes 3 split pictures, wherein the moni- 

this case, video signals can be transmitted as they are if a toring picture of the river shown in FIG. 2 A is deleted and 

high-speed transmission channel can be used as a transmis- the monitoring picture of the factory is arranged at the 

sion path. But normally, a low-speed transmission channel is position of the monitoring picture of the river. FIG. 2C 

generally used as the transmission path, and the video shows a split monitoring screen which includes 4 split 

signals must be compressed, encoded and transmitted. The 20 pictures and includes in addition to the monitoring pictures 

systems may be categorized into a type which synchronizes of the power line and the factory the monitoring pictures of 

and processes the video signals from a plurality of picture the P"*«? volcano and the vol <; ano which is picked up and 

pickup apparatuses, and a type which processes asynchro- recorded in the past, so as to facilitate comparison of the 

nous video signals. The system of the latter type has a P' esent volcano and , the P ast volcano Further, FIG. 2D 

simpler and less expensive construction. Accordingly, there 25 sh ° ws a ^ ^on^mg screen which includes 4 split 

, i , A- ■ ,i n- i • / c pictures and includes in addition to the monitoring pictures 

are demands to efficiently compose multiple pictures of r -., ,. j tU r 4 4t _ . • * c 

, i-i or the power line and the factory the monitoring pictures of 

asynchronoas video signals. (fae and m en]arged ^ of ^ volcan V such as a 

FIG. 1 is a diagram for explaining a synchronous video crater thereof, 

signal composing remote monitoring system which includes 3Q In order t0 use a com mon picture transmission unit with 

television cameras 201, a multiple picture composing appa- respect to a plurality of terminal equipments, a system which 

ratus 202 having a synchronizing signal distributor 202a, a transmits a composed multiple picture and separates the 

transmission unit 203 having an encoding function, a net- pictures at the receiving end to be transferred to correspond- 

work 204 such as a radio satellite line, ISDN, ATM network ing terminal equipments is proposed in a Japanese Laid- 

and LAN, a transmission unit 205 having a decoding 35 Open Patent Application No.4-7990, for example. In 

function, and a monitor 206. addition, a system which writes asynchronous video signals 

The 4 television cameras 201 respectively pickup pictures in a common memory by carrying out an address control and 

of objects to be monitored, namely, a volcano, a river, a composes the pictures which are divided along a vertical 

power line and a factory, by carrying out horizontal and direction of the common memory is proposed in a Japanese 

vertical scans based on synchronizing signals from the 40 ^aid-Open Patent Application No.54-148426, for example, 

synchronizing signal distributor 202a. Hence, video signals .The process of composing synchronized video signals is 

#1, #2, #3 and #4 which are output from the television ^lively easy since the video signals are synchronized, 

cameras 201 by picking up the corresponding objects to be However ' lhere » a P ro K b ^ m m that ' he m u ulUple plCtUfe 

monitored are synchronized, and the multiple picture com- ^LTe^ensive 5 CameS * C ° mP ° Smg 

posing apparatus 202 composes the video signals #1 through it P"^^ 15 ex P er ^ ive : 

uj i j *u u ■ ■ i * • * j 45 On the other hand, the process of composing asynchro- 

#4 based on the synchronizing signals to input a composed flized vkJeo gi ^ £ ^ J * J & 

video signal to the transrmssion unit 203. pictur T composing apparaUls a y s ^p^d t0 the 

The transmission unit 203 compresses, encodes and trans- casc where tnc synchronized video signals arc composed, 

mils the composed video signal depending on a transmission However, it is difficult to control the composing process in 

band and the like of the network 204. The transmission unit 50 such a mu i t i p i e picture composing apparatus. Moreover, in 

205 receives the compressed and encoded composed video the case of the proposed system which uses a common 

signal via the network 204, and restores the original com- me mory, the composed picture is divided along the vertical 

posed video signal by carrying out a process complementary direction> ^ the application of the system is limited, 

to that at the time of the compression and encoding. The , n addhi ^ h ^ standardized mcans described 

restored original composed video signa is input to the 55 above prescribed for the transmission system of the video 

monitor 206, and the monitor 206 displays a composed &{ , tQ ^ ^ ^ ^ a UcaMe t0 the 

picture of the object to the monitored, that is, the volcano, cncoding and decoding> * her £ is a piM * m in that the 

river, power line and tactory. standardized means cannot be applied automatically to the 

The multiple picture composing apparatus 202 has a mV Lltiple picture composing process. In other words, in the 

construction applied with a video editing function of a 60 case of the mu i tip i e p i ctur e composing process, the impor- 

television broadcasting .s^st.ejii^ancUncjMdes tire synchro,- tance of tne picture may among tbe pictur es, and a 

nizing signal distributor 202a. Although high-performance sufficient monitoring cannot be realized by composing all of 

multiple picture composing apparatuses have been reduced the p j clures i n the same manner, 
to practice, such multiple picture composing apparatuses are 

expensive. In addition, the transmission units 203 and 205 65 SUMMARY OF THE INVENTION 

respectively have a construction capable of compressing, Accordingly, it is a general object of the present invention 

encoding and decoding the video signal in accordance with to provide a novel and useful multiple picture composing 
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method and multiple picture composing apparatus, in which 
the problems described above are eliminated. 

Another and more specific object of the present invention 
is to provide a multiple picture composing method for 
composing video signals from a plurality of video signal 
sources which operate asynchronously to each other, com- 
prising the steps of (a) successively selecting the video 
signals from the video signal sources, (b) reading the 
selected video signals in units of frames while maintaining 
horizontal and vertical synchronization of the selected video 
signals, (c) reducing the video signals in the units of frames 
in correspondence with areas allocated for each of pictures 
to be composed in a composed picture, (d) writing the 
reduced video signal in blocks of a picture composing 
memory corresponding to the areas allocated for each of the 
pictures to be composed, and (e) composing the video 
signals stored in the picture composing memory to obtain a 
composed video signal related to a composed picture and 
subjecting the composed video signal to at least one of an 
encoding process and a transmission process. According to 
the multiple picture composing method of the present 
invention, it is possible to reduce the cost of the system by 
using asynchronous video signal sources. In addition, it is 
possible to simplify control operations by relating the pic- 
ture composing process and the encoding process or the 
transmission process. 

Still another object of the present invention is to provide 
a multiple picture composing apparatus for composing video 
signals from a plurality of video signal sources which 
operate asynchronously to each other, comprising a video 
signal selector selecting the video signals from the video 
signal sources, an input signal monitoring unit reading the 
video signals selected by the video signal selector while 
maintaining horizontal and vertical synchronization of the 
selected video signals, a horizontal direction reducer reduc- 
ing the video signals synchronized by the input signal 
monitoring unit in units of frames in correspondence with 
areas allocated for each of pictures to be composed in a 
composed picture, a picture composing memory storing the 
video signals reduced by the horizontal direction reducer in 
units of lines, a picture composing controller controlling 
reading of the video signals by the input signal monitoring 
unit via the video signal selector and a picture composing 
process with respect to the video signals stored in the picture 
composing memory, and an encoder carrying out the picture 
composing process with respect to the video signals stored 
in the picture composing memory. According to the multiple 
picture composing apparatus of the present invention, it is 
possible to reduce the cost of the system by using asynchro- 
nous video signal sources. In addition, it is possible to 
simplify control operations by relating the picture compos- 
ing process and the encoding process or the transmission 
process. 

Furthermore, if the setting of the quantization value, the 
number of times the video signals are read, the encoding 
process or the like are carried out with priority with respect 
to the video signal of a weighted channel, it is possible to 
improve the sharpness or clarity of an important part of the 
received and displayed composed picture, and the accuracy 
of a remote monitoring or the like can be improved. It is also 
possible to utilize an invalid area on the composed picture, 
so as to make an enlarged display, a zoom-up display, and 
returning of a specific pattern or received and decoded 
picture. Moreover, it is possible to carry out a remote control 
of the multiple picture composing process by exchanging 
information for the remote control operation between a 
monitoring end and a monitored end. Accordingly, the 
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present invention can be applied to various systems to 
realize an economical and efficient multiple picture com- 
posing process. 

Other objects and further features of the present invention 
5 will be apparent from the following detailed description 
when read in conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

10 FIG. 1 is a diagram for explaining a synchronous video 
signal composing remote monitoring system; 

FIGS. 2A through 2D respectively are diagrams showing 
monitoring screens at receiving ends of a remote monitoring 
system for explaining a multiple picture composing; 
35 FIG. 3 is a diagram for explaining an asynchronous video 
signal composing remote monitoring system; 

FIG. 4 is a diagram for explaining the relationship 
between a transmission band and a frame rate; 

FIG. 5 is a system block diagram showing a first embodi- 
ment of a multiple picture composing apparatus according to 
the present invention; 

FIGS. 6Aand 6B respectively are diagrams for explaining 
a reducing process; 
25 FIG. 7 is a flow chart for explaining an initializing 
process; 

FIGS. 8 A through 8D respectively are diagrams for 
explaining the arrangement of pictures in the composed 
picture; 

30 FIG. 9 is a flow chart for explaining an interrupt process; 
FIG. 10 is a flow chart for explaining the interrupt 
process; 

FIG. 11 is a flow chart for explaining a picture composing 
35 process of the first embodiment; 

FIG. 12 is a diagram for explaining the composing of the 
asynchronous video signals; 

FIG. 13 is a flow chart for explaining the picture com- 
posing process of a second embodiment of the multiple 
40 picture composing apparatus according to the present inven- 
tion; 

FIG. 14 is a flow chart for explaining the picture com- 
posing process of a third embodiment of the multiple picture 
composing apparatus according to the present invention; 

45 

FIG. 15 is a flow chart for explaining an interrupt process 
of the third embodiment; 

FIG. 16 is a flow chart for explaining the interrupt process 
of the third embodiment; 
50 FIGS. 17A and 17B respectively are diagrams for explain- 
ing reading frames; 

FIGS. 18A and 18B respectively are diagrams for explain- 
ing encoding frames; 

FIGS. 19Aand 19B respectively are diagrams for explain- 
55 ing receiving and displaying frames; 

FIG. 20 is a system block diagram showing a fourth 
embodiment of the multiple picture composing apparatus 
according to the present invention; 
6Q FIG. 21 is a flow chart for explaining a picture composing 
process of the fourth embodiment; 

FIG. 22 is a flow chart for explaining a video input 
judging process of the fourth embodiment; 

FIG. 23 is a flow chart for explaining an invalid area 
65 process of the fourth embodiment; 

FIG. 24 is a flow chart for explaining another invalid area 
process; 
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FIG. 25 is a flow chart for explaining still another invalid function, a network 214 such as a radio satellite line, ISDN, 

area process; ATM network and LAN, a transmission unit 215 having a 

FIGS. 26A through 26D respectively are diagrams for decoding function, and a monitor 216. 

explaining the arrangement of pictures in the composed r Ilie 4 television cameras 211 respectively pickup pictures 

picture; 5 °^ °bj ects to be monitored, namely, a volcano, a river, a 

' _ . „ , r , . . power line and a factory, independently. Hence, asynchro- 

27 » » flow chart for ex P lainin S an inva » d area ^ videQ sigQals #1> ^ #3 ^ #4 ^ afe ^ from 

process including a zoom-up process; the television came ras 211 by picking up the corresponding 

FIG. 28 is a flow chart for explaining the invalid area objects to be monitored are input to the asynchronous 

process including the zoom-up process; 1Q multiple picture composing apparatus 213a of the transmis- 

FIG. 29 is a flow chart for explaining a picture composing s * on unit 213, and the asynchronous multiple picture corn- 
process of a fifth embodiment of the multiple picture com- posing apparatus 213a composes the video signals #1 
posing apparatus according to the present invention; through #4. The transmission unit 213 compresses and 

FIG. 30 is a diagram for explaining CIF; enc0des the composed video signal depending on a trans- 

m ^ . , , , ... _ mission band of the network 214, and transmits the com- 

FIG. 31 is a system block diagram tor explaining an 15 p ressed and encoded composed video signal to the network 

encoder part of the fifth embodiment; 2U ^ transmission unit 2 \$ receives the compressed and ^ 

FIG. 32 is a flow chart for explaining a quantization value encoded composed video signal via the network 214, and / 

determining process of the fifth embodiment; restores the original composed video signal by carrying out / 

FIG. 33 is a diagram for explaining a video signal source 2Q a process which includes decoding and is complementary to I 

information obtaining unit; the process at the time of the compression and encoding. The \ 

FIGS. 34A through 34D respectively are diagrams for restored original composed video signal is input to the \ 

explaining motion estimation and arrangement of pictures of monitor 216, and the monitor 216 displays a composed \ 

the composed picture- picture of the objects to the monitored, that is, the volcano, \ 

FIG. 35 is a diagram for generally explaining the remote 25 ri u ver > P ower ^ e and factory as shown in FIG. 1 described J 

control in a sixth embodiment of the multiple picture com- above or ™ shoWD 10 anv of FIGS * 2A ihrou & 2D * ^ 

posing apparatus according to the present invention; If the transmission band of the network 214 is the same as 

T7T.no «* «n Aito .-I a ' c that of the network 204 shown in FIG. 1, the compression, 

FIGS. 36A, 36B and 36C respectively are diagrams for , , A r ., * . . 

... ' . i -j encoding and decoding functions of the transmission units 

explaining a remote control guidance; j*** i_ ,? iL r*i_ * 

r & & 30 213 and 215 may be the same as those of the transmission 

FIG. 37 is a diagram for explaining a picture switching by units 20 3 and 205 shown in FIG. I. Since the transmission 

the remote control; umt 2 13 carries out an asynchronous composing process, the 

FIG. 38 is a time chart for explaining a picture composing multiple picture composing apparatus 202 having the syn- 

mode determination sequence; chronizing signal distributor 202a shown in FIG. 1 may be 

FIG. 39 is a diagram for explaining a remote control 35 omitted in FIG. 3, In addition, because the video signals 

protocol; from the television cameras 211 can be composed by the 

FIGS. 40A and 40B respectively are diagrams for explain- asynchronous multiple picture composing apparatus 213a 

ing a single video signal source selection command and a which * within ^ transmission unit 213 and has a relatively 

single video signal source selection response; sim P le construction, the system shown in FIG. 3 is less 

t-t/^o a j im t - 1 j ■ c , - d n expensive compared to the system shown in FIG. 1. 

FIGS. 41 A and 41B respectively are diagrams for explain- 40 V,,^ M . r r J , . . , , . 

ing a multiple picture composing set command; 4 1S a dia S ram explaining the relationship 

_Z ^ r _„ t A ^ . , between a transmission band and a frame rate. For example, 

FIGS. 42A, 42B, 42C, 42D and 42E respectively are if tne frame rflte of me ^ s{ d is 3Q frames/seC) the 

diagrams for explaining a multiple picture composing transmission path requires a band of 6 Mbps. Hence, a frame 

response, a multiple picture composing inquiry command, a fate of oximatd 2 fo mcs/scc * require d in a case 

multiple picture composing inquiry response, a video signal wherc the videQ si { subjected t0 various ^ of com . 

source state request command and a video input source state pression afld encoding [& transmitted Wa the transmission 

request response; and path having a band of 64 kbps sucn ^ a B . channel of the 

FIGS. 43A, 43B, 43C and 43D respectively are diagrams ISDN, In the remote monitoring system, video conference 

for explaining a remote guidance set/cancel command, a 5Q system or the like described above, the picture which is 

remote guidance set/cancel response, a video signal abnor- picked up includes relatively small movements. For this 

mality indication and a multiple picture composing modifl- reason, even if the encoded video signal is transmitted at a 

cation indication. low frame rate and is reproduced and displayed at the 

OFSPRTPTinN of thf prfffrrfd receiving end, the quality of the picture display is sufficient 

DESCRIPTION OF raEPREFERRED ^ fof tfae purposes of the mon it or ing the remote picture, the 

hMUUUlMhN lb participants of the conference or the like. 

First, a description will be given of an asynchronous video FIG. 5 is a system block diagram showing a first embodi- 

signal composing remote monitoring system to which the ment of a multiple picture composing apparatus according to 

present invention may be applied. FIG. 3 is a diagram for the present invention. More particularly, FIG. 5 shows an 

explaining the asynchronous video signal composing remote 60 important part of the transmission unit 213 shown in FIG. 3 

monitoring system, and FIG. 4 is a diagram for explaining which composes and encodes the asynchronous video 

the relationship between a transmission band and a frame signals, together with television cameras. This embodiment 

rate. of the multiple picture composing apparatus employs a first 

The asynchronous video signal composing remote moni- embodiment of a multiple picture composing method 

loring system shown in FIG. 3 includes television cameras 65 according to the present invention. 

211, a transmission unit 213 having an asynchronous mul- In FIG. 5, television cameras 1-1 through 1-n are made up 

tiple picture composing apparatus 213a and an encoding of video signal sources including digital cameras, recording 
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and/or reproducing apparatuses and the like. The transmis- 
sion unit 213 includes a video signal selector 2, an input 
signal monitoring unit 3, a horizontal direction reducer 4, a 
line memory 5 which is made up of a first-in-first-out 
(FIFO), a picture composing memory 6, an encoding 5 
memory 7, an encoder 8, a picture composing controller 9, 
an encoding controller 10, an interrupt handler 11, an 
interrupt controller 12, a first direct memory access control- 
ler (DMAC) 13, and a second DMAC 14 which are coupled 
as shown in FIG. 5. 10 

The television cameras 1-1 through 1-n are independently 
synchronized, and pickup pictures of objects to be moni- 
tored. The video signal selector 2 successively selects video 
signals from the television cameras 1-1 through 1-n under a 
control of the picture composing controller 9, and inputs the 15 
selected video signal to the input signal monitoring unit 3. 
The video signals from the television cameras 1-1 through 
1-n may employ any of various kinds of systems such as the 
NTSC system, the PAL system and the SECAM system. In 
addition, the video signals may be made up of an analog 20 
composite signal, an analog component signal including 
luminance and color signals, a digital composite signal 
which is obtained by digitizing an analog video signal, a 
digital component signal including a luminance signal Y and 
color difference signals Cb and Cr, or the like. 25 

FIG. 5 shows a state where the video signal from the 
television camera 1-2 is selected by the video signal selector 
2. The input signal monitoring unit 3 monitors a vertical 
synchronizing signal V and a horizontal synchronizing sig- 
nal H of the video signal which is input from the video signal 30 
selector 2, and achieves vertical synchronization and hori- 
zontal synchronization so as to input the video signal 
amounting to 1 frame. When a vertical position specified 
from the interrupt controller 12 is detected, the input signal 
monitoring unit 3 notifies this detection of the specified 35 
vertical position to the interrupt controller 12. The interrupt 
controller 12 calls the interrupt handler 11 which is regis- 
tered in advance. 

The first DMAC 13 transfers the data within the line 4Q 
memory 5 to a specified memory access in response to a 
request signal from the line memory 5, and in this case, the 
data is transferred to the picture composing memory 6. After 
the transfer of the data amounting to 1 line to the picture 
composing memory 6 ends, the memory address in the 45 
vertical direction is incremented by a specified number of 
lines. On the other hand, the second DMAC 14 transfers the 
data from the specified memory address to another specified 
memory address in response to an instruction from the 
picture composing controller 9. 5Q 

The horizontal direction reducer 4 decimates or thins 
pixels on the line so as to reduce the picture in the horizontal 
direction, and the data related to 1 reduced line is written 
into the line memory 5. When the data amounting to 1 line 
which is reduced in the horizontal direction is written into 55 
the line memory 5, a DMA request is supplied to the first 
DMAC 13. The first DMAC 13 carries out a control to 
transfer the data from the line memory 5 to the picture 
composing memory 6. 

The picture composing memory 6 is made up of blocks 60 
which divide the picture composing memory 6 depending on 
the number of pictures to be composed. The composed video 
signal related to the composed pictures is transferred to the 
encoding memory 7. The encoder 8 transmits the composed 
video signal stored in the encoding memory 7 after subject- 65 
ing the composed video signal to compression and encoding 
such as intra- frame encoding and inter- frame encoding. In 
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this case, the picture composing memory 6 and the encoding 
memory 7 may have a double buffer construction, so that the 
double buffer is switched to alternately realize the functions 
of the picture composing memory 5 and the encoding 
memory 7. In FIG. 5, the illustration of a transmitter which 
actually transmits the compressed and encoded video signal 
to the network 214 is omitted. 

FIGS. 6Aand 6B respectively are diagrams for explaining 
a reducing process. FIG. 6Ashows 1 picture made up of 705 
pixelsx525 lines. By subjecting the picture shown in FIG. 
6 A to a simple thinning process by thinning pixels indicated 
by hatching in FIG. 6B, it is possible to reduce the picture 
shown in FIG. 6A by V£ in both the vertical and horizontal 
directions. Accordingly, in the case where an effective area 
of the CCIR601 is 704 pixelsx480 lines, the picture is 
reduced to 352 pixelsx240 lines by the simple thinning 
process described above. The reduction in the vertical direc- 
tion can be made by reading only one of odd and even fields. 

FIG. 7 is a flow chart for explaining an initializing 
process. The initializing process shown in FIG. 7 initializes 
various parts shown in FIG. 5. A step Al masks the interrupt 
controller 12 and the DMACs 13 and 14 so as to disable the 
operations of the interrupt controller 12 and the DMACs 13 
and 14. A step A2 obtains a number n of video signal 
sources, that is, the television cameras 1-1 through 1-n in 
this case, and arrangement information related to positions 
on a screen where the composed pictures are to be displayed 
for the n video signal sources. The information obtained in 
the step A2 may be specified by the user or, set in the system 
in advance. 

A step A3 substitutes 1 into a block number A in the 
picture composing memory 6, where the block number A is 
an integer taking any value from 1 to n, and n«l to 4 in the 
case where the composed picture is made up of 4 pictures. 
A step A4 obtains a video signal source channel number C A , 
and a step A5 obtains an odd/even field read variable 0 A . A 
step A6 obtains an increment value V A of the picture 
composing memory 6, and a step A7 obtains a horizontal 
direction reduction value H A . A step A8 sets a picture 
composing start address XS A , YS A , that is, a horizontal 
direction read start position and a horizontal direction read 
end position within the picture composing memory 6. A step 
A9 sets a picture composing end address XE A , YE A , that is, 
a vertical direction read start position and a vertical direction 
read end position within the picture composing memory 6. 
In addition, a step A10 obtains interrupt generation condi- 
tions VS A and VE A . A step All decides whether or not A-n. 
If the decision result in the step All is NO, a step A12 
increments Aby 1, and the process returns to the step A4. On 
the other hand, the process ends if the decision result in the 
step All is YES, Hence, the initial setting is carried out for 
A=l to n. 

The increment value V A obtained in the step A6 is a value 
by which the DMAC 13 automatically increments the 
address in the vertical direction after the DMA transfer 
amounting to 1 line is completed when transferring the data 
from the line memory 5 to the picture composing memory 6. 
The odd/even field read variable 0 A obtained in the step A5 
indicates a condition used to judge whether only the even 
field or both the even and odd fields are to be read within the 
interrupt handler 11. The interrupt generation conditions 
VS A and VE A obtained in the step A10 are set as the 
conditions for generating the interrupt, and values of the 
interrupt generation conditions VS A and VE A may be set in 
advance depending on the system. Further, the video signal 
source channel C A obtained in the step A4 may be set by the 
user. For example, in a case where C 4 -2, it is indicated that 
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the video signal from the video signal source 2 is read into After the initializing process is completed, a video signal 

the 4th block within the picture composing memory 6. read process is started to read the video signals from the 

In the initializing process described above, if the number video signal sources. First, A is set to A=l, and Q is set in 

n of video signal sources is obtained, the arrangement the video signal selector 2. In this case, C a is set to C^l, and 

information depending on the number n is obtained simul- 5 thus, the video signal from the video signal source 1-1 is 

taneously therewith. Next, the XS A ,XE A , YS A , YE^H^, V A , selected by the video signal selector 2. In addition, the 

0 A , VS A and VE^ are obtained from the arrangements which increment value V 5 of the picture composing memory 6 is 

are successively obtained from A«l to A=n. For example, set in the DMAC 13, and the horizontal direction reduction 

the initial value of the video signal source channel number value is set in the horizontal direction reducer 4. At the 

C A is set to C A =A. 10 same time, a read start position XS l5 YS X of the picture 

FIGS. 8A through 8D respectively are diagrams for composing memory 6 is substituted into a transfer destina- 

explaining the arrangement of pictures in the composed tion address XO, YO. The transfer destination address XO, 

picture. FIGS. 8A through 8D show the arrangement of the YO is a variable which is used within the interrupt handler 

pictures for a case where n~l to 4, and it is assumed for the U wnen setting the transfer destination address in the 

sake of convenience that the areas of the composed picture DMAC 13. When the process described above is completed, 

correspond to the blocks of the picture composing memory 15 1 is substituted into a video signal read complete recognition 

6 where the video signals are written. The initial values variable FLAG, VSj is set as the interrupt generation 

described above for n-1 shown in FIG. 8A, that is, in a case condition, and the interrupt mask is cleared. In other words, 

where no picture composing process is carried out, are as me operation of the interrupt controller 12 is enabled, 

follows. Th e input signal monitoring unit 3 monitors the vertical 

20 synchronizing signal V of the video signal which is input 

XS^XS from the video signal source, that is, the television camera, 

XEi-XS+X via the video signal selector 2. When the VS r th line is 

YSj-YS reached by starting the count from the vertical synchronizing 

YE^YS+Y signal V, the input signal monitoring unit 3 inputs an 

H a =l 25 interrupt signal to the interrupt controller 12. The interrupt 

V 1 »2 controller 12 calls the interrupt handler 11 in response to the 

0 3 =1 interrupt signal. 

^i m ^ start FIGS. 9 and 10 are flow charts for explaining an interrupt 
VE, 

-vend process. The flow charts shown in FIGS. 9 and 10 corre- 

30 spond to the interrupt process of the interrupt handler 11. A 

In addition, the initial values described above for n«2 to description will hereinafter be given separately for the case 

4 shown in FIGS. 8B through 8D, are as follows, where where n=l and for the case where n=2 to 4. 

VSTART indicates a start address of a video effective region First, in the case where n=l, a step Bl shown in FIG. 9 

of the video signal source, and VEND indicates an end decides whether or not the video signal read complete 

address of the video effective region of the video signal 35 recognition variable FLAG is 1. If the decision result in the 

source. step Bl is YES, a step B2 decides whether or not the field 

is an even field. If the decision result in the step B2 is YES, 

XS 1 =XS 2 «XS a step B3 sets the transfer destination address XO, YO in the 

XS 3 =XS 4 =XS+X/2 DMAC 13, and a step B6 clears the mask of a DMA1 of the 

XE 3 =XE 2 -XE+X/2 40 DMAC 13. A step B7 sets VB 1 as the next interrupt 

XE 3 -XE 4 =XS+X generation condition, a step B8 substitutes 2 into the video 

YS 1 =YS 3 -YS signal read complete recognition variable FLAG, and the 

YS 2 =YS 4 =YS+Y/2 process ends. 

YE 1 «YE 4 =YS+Y/2 On the other hand, if the decision result in the step Bl is 

YE 2 =YE 3 =YS+Y 45 NO, a step B9 shown in FIG. 10 decides whether not the 

H 1 =H 2 =H 3 =H 4 =l/2 video signal read complete recognition variable FLAG is 2. 

V 1 =V 2 «V 3 =V 4 =1 If the decision result in the step B9 is YES, a step B10 masks 

0^02=03=04=0 the DMA1 of the DMAC 13. A step Bll decides whether or 

VS 1 «VS 2 =VS 3 =VS 4 =V ATA/er not the odd/even field read variable 0 A is 1. If the decision 

VE 1 =VE 2 =VE 3 » VE 4 =V £iV£> 50 result in the step Bll is NO, a step B15 substitutes 0 into the 

video signal read complete recognition variable FLAG, and 

In the case shown in FIG. 8 A where n=l, that is, when no the process ends. On the other hand, if the decision result in 

picture composing process is carried out, both the odd field the step Bll is YES, a step B12 decides whether or not the 

and the even field are read, and the odd and even fields are reading of the odd and even fields is completed, and the 

alternately stored in the picture composing memory 6. 55 process advances to the step B 15 if the decision result in the 

Hence, the increment value V A becomes 2, and the odd/even step B12 is YES. If the decision result in the step B12 is NO, 

field read variable 0 A becomes 1. In addition, in the cases a step B13 substitutes 1 into the video signal read complete 

shown in FIGS. 8B through 8D where n-2 to 4, that is, when recognition variable FLAG. In addition, a step B14 sets the 

the picture composing process is carried out, only the even interrupt generation condition VS A , and the process ends, 

fields are read and consecutively stored in the picture 60 If the decision result in the step B2 shown in FIG. 9 is NO, 

composing memory 6. This, the increment value V A a step B4 decides whether or not the odd/even field read 

becomes 1, and the odd/even field read variable 0 A becomes variable 0 A is 1. If the decision result in the step B4 is YES, 

0. In the latter cases, the reduction in the horizontal direction a step B5 sets the transfer destination address to XO, YO+1 , 

is carried out by a hardware structure of the horizontal and the process advances to the step B6. On the other hand, 

direction reducer 4. On the other hand, the reduction in the 65 if the decision result in the step B4 is NO, the process ends, 

vertical direction is realized by reading only one of the odd More particularly, the video signal is input to the line 

and even fields. memory 5 via the horizontal direction reducer 4. The reduc- 
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tion value of the horizontal direction reducer 4 is 1; and the 
reducing process is not carried out with respect to the video 
signal. When data amounting to 1 line is stored in the line 
memory 5, a DMA request signal is supplied to the DMAC 
13. The DMAC 13 transfers the data amounting to 1 line and 
stored in the line memory 5 to the transfer destination 
address XO, YO, and YO of the transfer destination address 
is incremented by 2. 

Next, when data amounting to 1 line is stored in the line 
memory 5, a DMA request signal is again supplied to the 
DMAC 13. Hence, the data amounting to 1 line and stored 
in the line memory 5 is transferred to the transfer destination 
address XO, YO+2, and YO of the transfer destination 
address is incremented by 2. Accordingly, the data amount- 
ing to 1 line is stored for every other vertical direction 
address YO of the picture composing memory 6. In addition, 
because the interrupt generation condition VS 3 indicates a 
starting line of the effective region of the video signal, the 
data transfer is started from the start of the video signal from 
the video signal source. 

The input signal monitoring unit 3 receives the video 
signal from the video signal source via the video signal 
selector 2. When the VEj-th line is reached by starting the 
count from the vertical synchronizing signal V of the video 
signal, the input signal monitoring unit 3 notifies an interrupt 
signal to the interrupt controller 12, and the interrupt con- 
troller 12 calls the interrupt handler 11 depending on the 
interrupt generation condition. 

In this state, FLAG«2 since 2 is substituted into the video 
signal read complete recognition variable FLAG in the step 
B8 shown in FIG. 9. In addition, the decision result in the 
step Bl shown in FIG. 9 is NO. Hence, the process advances 
to the step B9 shown in FIG. 10 and the video signal read 
complete recognition variable FLAG becomes FLAG=2. As 
a result, the interrupt handler 11 masks the DMAC 13 and 
the operation of the DMAC 13 is ended. The odd/even field 
read variable is 1 and the even field is read, but the odd 
field is not read. Accordingly, 1 is again substituted into the 
video signal read complete recognition variable FLAG in the 
step B13 shown in FIG. 10, the interrupt generation condi- 
tion VE a is set in the step B14, and the process ends. The 
interrupt generation condition VE ± indicates the ending line 
of the effective region of the video signal, and thus indicates 
the completion of the data transfer amounting to 1 field of 
the video signal from the video signal source. 

When the VSj-th line is reached by starting the count 
from the vertical synchronizing signal V of the video signal 
from the video signal source, the interrupt handler 11 is 
called. In this state, FLAG-l in the step Bl and the data of 
the off field is being input in the step B2. Hence, the transfer 
destination address XO, YO+1 is set in the DMAC 13 in the 
step B5, and the masking of the DMA1 of the DMAC 13 is 
cleared in the step B6. Then, the next interrupt generation 
condition VE 1 is set in the step B7, 2 is substituted into the 
video signal read complete recognition variable FLAG in the 
step B9, and the process ends. 

When the DMAC 13 receives the DMA request signal, the 
DMAC 13 transfers the data amounting to 1 line and stored 
within the line memory 5 to the transfer destination address 
XO, YO+1, and increments the transfer destination vertical 
direction address YO by 2. In other words, the next data 
amounting to the 1 line is stored at the transfer destination 
address XO, YO+3. Hence, the data are successively stored 
at shifted locations in the picture composing memory 6 so as 
to form pairs with the data of the even fields. 

Similarly as described above, when the VE^th line is 
reached by starting the count from the vertical synchronizing 
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signal V of the video signal from the video signal source, the 
interrupt handler 11 is called. In this state, FLAG-2 and the 
DMA1 of the DMAC 13 is masked, so as to end the 
operation of the DMAC 13. In addition, since O-^l in the 

5 step Bll shown in FIG. 10 and only the odd field is read in 
the step B12, 0 is substituted into the video signal read 
complete recognition variable FLAG in the step B15, and 
the process ends. Further, because the interrupt generation 
condition VE 1 indicates the ending line of the effective 

10 region of the video signal, the completion of the data transfer 
amounting to 1 frame of the video signal from the video 
signal source is indicated. 

On the other hand, in the case where n-2 to 4, when 
FLAG-1 in the step Bl and the data of the odd field is 

15 presently being input, the process of the interrupt handler 11 
is ended immediately so as to wait for the input of the data 
of the even field. In addition, when the data of the even field 
is being input, the transfer destination address XO, YO is set 
in the DMAC 13 in the step B3, and the masking of the 

20 DMA1 of the DMAC 13 is cleared in the step B6. Then, the 
next interrupt generation condition VEj is set in the step B7, 
2 is substituted in the video signal read complete recognition 
variable FLAG in the step B8, and the process ends. 
In this state, if the reduction value is l A> the video signal 

25 which is thinned by Vi in the horizontal direction reducer 4 
is input to the line memory 5. As soon as the data amounting 
to 1 line is stored in the line memory 5, the DMA request 
signal is supplied with respect to the DMAC 13. When the 
DMAC 13 receives the DMA request signal, the DMAC 13 

30 transfers the data amounting to 1 line stored in the line 
memory 6 to the transfer destination address XO, YO, and 
increments the vertical direction address YO by 1. 

When the next data amounting to 1 line is stored in the 
line memory 5, a DMA request signal is supplied again to the 

35 DMAC 13. As a result, the transfer destination address 
becomes XO, YO+1, and the data amounting to 1 line is 
transferred from the line memory 5. Then, the vertical 
direction address YO is incremented by 1. Further, because 
the interrupt generation condition VS 1 indicates the starting 

40 line of the effective region of the video signal, the start of the 
data transfer from the start of the video signal from the video 
signal source is indicated. 

Similarly as described above, when the VEj-th line is 
reached by starting the count from the vertical synchronizing 

45 signal V of the video signal from the video signal source, the 
interrupt handler 11 is called. In this state, FLAG=*2 in the 
step B9 and the DMA1 of the DMAC 13 is masked in the 
step B10, so as to end the operation of the DMAC 13. In 
addition, since O-^O, 0 is substituted in the video signal read 

50 complete recognition variable FLAG in the step B15, and 
the process ends. Further, when the interrupt generation 
condition VE 1 indicates the ending line of the effective 
region of the video signal, the horizontal direction of the 
video signal from the video signal source is reduced by 

55 and the transfer of the data amounting to 1 field is com- 
pleted. 

While the above described process is being carried out by 
the interrupt handler 11, the picture composing controller 9 
monitors the value of the video signal read complete rec- 

60 ognition variable FLAG, and waits until the value becomes 
0. When the value of the video signal read complete recog- 
nition variable FLAG becomes 0, the reading of the video 
signal amounting to 1 frame is completed. Hence, the 
interrupt is masked and A=l is updated to A«2, so as to 

65 repeat the process described above. 

The above described process is carried out for A-l to 
A=n, and upon completion of the process, the reading of the 
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data amounting to 1 frame into the picture composing 
memory 6 is completed, With respect to the data from the 
picture composing memory 6, the encoding memory 7 and 
the encoder 8 carry out an encoding process under the 
control of the encoding controller 10, and a transmission 
process is carried out to transmit the encoded video signal to 
the network 214. The encoding process and/or the transmis- 
sion process depend on the system, and for example, the 
encoding process may be the H.261, H.263, MPEG1, JPEG 
or the like, while the transmission process may be with 
respect to the low-speed LAN for the bit map data, low- 
speed point-to-point protocol (PPP) or the like. Upon 
completion of the encoding process and/or the transmission 
process, the video signal is read again for A»l to A«n, so as 
to repeat the above described process. If the video signal is 
read for A=l to A=4 as described above, the composed 
picture obtained as a result of the picture composing process 
becomes as shown in FIGS. 2A, 2C and 2D, for example. 

FIG, 11 is a flow chart for explaining a picture composing 
process of the first embodiment. First, a step CI carries out 
the initializing process shown in FIG. 7. A step C2 substi- 
tutes 1 into A, and a step C3 sets C A to the video signal 
selector 2 shown in FIG. 5. A step C4 sets the increment 
value V A of the picture composing memory 6, and a step C5 
sets the horizontal direction reduction value H A in the 
horizontal direction reducer 4. A step C6 substitutes XS A , 
YS A into the picture transfer destination address XO, YO. A 
step C8 sets the interrupt generation condition VS A , and a 
step C9 clears the interrupt mask. 

A step CIO decides whether or not FLAG«0, and a step 
Cll carries out the interrupt mask if the decision result in the 
step CIO is YES. A step C12 decides whether or not A=n, 
and if the decision result in the step C12 is YES, the reading 
is completed and a step C13 carries out the encoding process 
and/or the transmission process. Astep C14 increments A by 
1, and if A>n, 1 is substituted into A and the process returns 
to the step C3. On the other hand, if the decision result in the 
step C12 is NO, the process advances to the step C14 
described above. Accordingly, n asynchronous video signals 
are composed, encoded and transmitted to the receiving end. 

FIG. 12 is a diagram for explaining the composing of the 
asynchronous video signals. In FIG. 12, video signals #1 
through #4 are asynchronous, and (1), (2), (3), . . . indicate 
frame numbers. When composing the video signals #1 
through #4 in this order, the composed video signal becomes 
#1(1), #2(2), #3(4) and #4(5), and although the illustration 
of the next video signal forming the composed video signal 
is omitted, the next video signal is #1(7). The frame rate of 
one video signal is 29.97 Hz, and the frame rate at which the 
reading is made as a whole in the case described above is 
inbetween 29.97 Hz and 14.985 Hz which is V4 of 29.97 Hz. 
In addition, in the case of the composed picture made up of 
4 pictures, the frame rate at which the video signal #1 is read 
is between 7.4925 Hz which is Vi the frame rate of 29.97 Hz 
and 3.74625 Hz which is Vi of 7.4925 Hz. The read time per 
frame is (1/29.97) sec to (2/29.97) sec. 

It is necessary to wait for the start of the frame when 
composing such asynchronous video signals. For example, 
when composing 9 video signals to obtain a composed 
picture made up of 9 pictures, a wait time of approximately 
300 msec to 600 msec is generated, and the encoding 
process and/or the transmission process cannot be carried 
out during this wait time. In the case of a still picture 
transmission which requires an extremely long time of 1 sec 
to several tens of sec to encode 1 frame or a transmission via 
a network having a narrow communication band of approxi- 
mately 64 kbps or less, for example, it is possible to 
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sufficiently apply the multiple picture composing means of 
the first embodiment shown in FIG. 5 to compose the 
asynchronous video signals. However, the wait time 
becomes a problem in the case of a transmission via a 

5 network having an intermediate transmission band of 
approximately 64 kbps to 1.5 Mbps which completes the 
encoding of 1 frame in several tens of msec to several 
hundreds of msec, b 

FIG. 13 is a flow chart for explaining the picture com- 

10 posing process of a second embodiment of the multiple 
picture composing apparatus according to the present inven- 
tion. This embodiment of the multiple picture composing 
apparatus employs a second embodiment of a multiple 
picture composing method according to the present inven- 

is tion. Because the important part of the transmission unit 213 
shown in FIG. 3 which composes and encodes the asyn- 
chronous video signals may have the same construction as 
that shown in FIG. 5, the illustration thereof will be omitted. 
In FIG. 13, steps Dl through D10 respectively correspond 

20 to the steps CI through C10 shown in FIG. 11, and a 
description thereof will be omitted. In the first embodiment 
shown in FIG. 11, after the reading of the video signal from 
the video signal source 1-1 is completed by setting A to A^l, 
A is successively set to A=2, 3, . . . , n, so as to read the video 

25 signals from the video signal sources 1-2 through 1-n. In 
addition, the encoding process and/or the transmission pro- 
cess are carried out after reading of 1 frame of the video 
signal from each of the video signal sources 1-1 through 1-n 
is completed. 

30 On the other hand, in this second embodiment, steps Dll 
through D13 are carried out, and by completing the reading 
of the video signal from the video signal source 1-1 for A«l, 
for example, the step D12 carries out an encoding process 
and/or a transmission process. The step Dll, prior to the step 

35 D12, carries out an interrupt, similarly to the step Cll shown 
in FIG. 11. Then, the step D13 increments A by 1, and 1 is 
substituted into A if A>n. Hence, the video signal from the 
video signal source 1-2 is read for A«2, and the encoding 
process and/or the transmission process are carried out upon 

40 completion of this reading of the video signal from the video 
signal source 1-2. The above described process is repeated 
until A becomes A-n, and when A becomes A>n, A is set to 
A»l in the step D13 and the process returns to the step D3. 
Accordingly, the initial values are stored in blocks of the 

45 picture composing memory 6 shown in FIG. 5 other than the 
block which stores the video signal from the video signal 
source 1-1 for A=l. After the encoding process and/or the 
transmission process are completed with respect to the video 
signal from the video signal source 1-1, A is set to A=2, and 

50 the reading of the video signal from the video signal source 
1-2 for A=2 is started. Upon completion of this reading of 
the video signal from the video signal source 1-2, the data 
stored in the picture composing memory 6 is subjected to an 
encoding process and/or a transmission process. 

55 At this point in time, the initial values are stored in the 
blocks of the picture composing memory 6 other than the 
blocks which store the video signals from the video signal 
sources 1-1 and 1-2. In addition, the block of the picture 
composing memory 6 storing the video signal from the 

60 video signal source 1-1 has the same content as that at the 
time of the previous encoding process and/or the transmis- 
sion process. Hence, by repeating the process for A«l to 
A=n, the contents of the picture composing memory 6 are 
successively updated for every 1 block. In this case, even if 

65 the number n of the video signal sources is set to a large 
value, the wait time of the picture composing process using 
the asynchronous video signals falls in a range of appro xi- 
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mately 33 msec to 66 msec, and this embodiment is suffi- 
ciently applicable to a system which carries out the encoding 
process and/or the transmission process at a rate of 15 
frames/sec. 

FIG. 14 is a flow chart for explaining the picture com- 5 
posing process of a third embodiment of the multiple picture 
composing apparatus according to the present invention. 
This embodiment of the multiple picture composing appa- 
ratus employs a third embodiment of the multiple picture 
composing method according to the present invention. 10 
Because the important part of the transmission unit 213 
shown in FIG. 3 which composes and encodes the asyn- 
chronous video signals may have the same construction as 
that shown in FIG, 5, the illustration thereof will be omitted. 

In FIG. 14, steps El through E7 respectively correspond is 
to the steps CI through C7 shown in FIG. 11 and the steps 
Dl through D7 shown in FIG. 13, and a description thereof 
will be omitted. This third embodiment includes a process of 
transferring the data from the picture composing memory 6 
to the encoding memory 7 by DMA, and additionally uses 20 
a video signal read complete recognition variable FLAG2. 

In other words, the step E7 substitutes 1 into the video 
signal read complete recognition variable FLAG, and a step 
E8 substitutes 1 into the video signal read complete recog- 
nition variable FLAG2. A step E9 sets an interrupt genera- 25 
tion condition VS^, and a step E10 clears the interrupt mask. 
A step Ell decides whether or not FLAG2-0, and a step E12 
carries out an interrupt mask if the decision result in the step 
Ell is YES. A step E13 starts a data transfer from the picture 
composing memory 6 to the encoding memory 7 by a DMA2 30 
of the DMAC 14, and a step E14 clears the interrupt mask, 
A step E15 carries out an encoding process and/or a trans- 
mission process, and the process returns to the step Ell. 

FIGS, 15 and 16 are flow charts for explaining an inter- 
rupt process of the third embodiment. In FIG. 15, steps Fl 35 
through F5 respectively correspond to the steps Bl through 
B5 shown in FIG. 9, and a description thereof will be 
omitted. In this embodiment, a step F6 substitutes 1 into the 
video signal read complete recognition variable FLAG2 
after the step F3 or F5 and prior to a step F7 which clears the 40 
mask of the DMA1 of the DMAC 13. Then, the mask of the 
DMA1 is cleared in the step F7. A step F8 sets an interrupt 
generation condition VE A , a step F9 substitutes 1 into the 
video signal read complete recognition variable FLAG, and 
the process ends. 45 

If the decision result in the step Fl is NO, a step F10 
shown in FIG. 16 decides whether or not FLAG =2. The 
process ends if the decision result in the step F10 is NO. On 
the other hand, if the decision result in the step F10 is YES, 
a step Fll masks the DMA1 of the DMAC 13, and a step 50 
F12 decides whether or not 0 A «1. If the decision result in 
the step F12 is YES, a step F13 decides whether or not the 
reading is completed, and a step F14 substitutes 1 into the 
video signal read complete recognition variable FLAG if the 
decision result in the step F13 is NO. A step F15 sets an 55 
interrupt generation condition VS A , and the process ends. 
These steps F10 through F15 respectively correspond to the 
steps B9 through B14 shown in FIG. 10. 

On the other hand, if the decision result in the step F12 is 
NO or the decision result in the step F13 is YES, the process 60 
advances to a step F16. For example, if 0^1 when A«l and 
the reading of the video signal amounting to 1 frame is 
completed or, if O^O and the reading of the video signal is 
completed, the interrupt handler 11 changes A from A=l to 
Ao2. More particularly, the step F16 increments A by 1, and 65 
substitutes 1 to A if A>n. A step F17 sets C A in the video 
signal selector 2. In this case, A=2, and Cj is set in the video 
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signal selector 2. As a result, the video signal selector 2 
selects the video signal from the video signal source 1-2. 

A step F18 sets an increment value V A of the picture 
composing memory 6, and a step F19 sets a horizontal 
direction reduction value H A in the horizontal direction 
reducer 4. A step F20 substitutes XS A , Y$ A into a picture 
transfer destination address XO, YO. In this case, the 
increment value set in the DMAC 13 is the reduction 
value set in the horizontal direction reducer 4 is H 2 , and XS lt 
YSj is substituted into the transfer destination address XO, 
YO. When the above described process is completed, a step 
F21 substitutes 0 into the video signal read complete rec- 
ognition variable FLAG2, and the process advances to the 
step F14. Hence, the step F14 substitutes 1 into the video 
signal read complete recognition variable FLAG, the step 
F15 sets an interrupt generation condition VS 2 , and the 
process ends. 

While the process is being carried out by the interrupt 
handler 11, the picture composing controller 9 monitors the 
value of the video signal read complete recognition variable 
FLAG2 and waits until the value becomes 0. When the value 
of the video signal read complete recognition variable 
FLAG becomes 0, the reading of the video signal amounting 
to 1 frame from the video signal source 1-1 is completed, 
and thus, the interrupt is masked, and the data transfer from 
the picture composing memory 6 to the encoding memory 7 
is thereafter carried out by the DMAC 14. Upon completion 
of this DMA transfer, the interrupt mask is cleared, and the 
encoding process and/or the transmission process is carried 
out by the encoder 8. 

While the encoding process and/or the transmission pro- 
cess are being carried out, the interrupt mask continues to 
remain in the cleared state. Hence, if an interrupt signal is 
received in this state, the interrupt handler 11 is called, and 
the video signal from the video signal source 1-2 next to the 
video signal source 1-1 is read into the picture composing 
memory 6. After this reading of the video signal is 
completed, the setting is next made to read the video signal 
from the video signal source 1-3. Furthermore, if an inter- 
rupt signal is received, the interrupt handler 11 is called, and 
this time, the video signal from the video signal source 1-3 
is read into the picture composing memory 6. By repeating 
such a process, the reading, encoding and.or transmission of 
the video signal are made, and the composed video signal in 
which the pictures are composed and encoded is transmitted 
to the receiving end. 

When the encoding process and/or the transmission pro- 
cess controlled by the picture composing controller 9 end, 
the value of the video signal read complete recognition 
variable FLAG2 is monitored. When the value of the video 
signal read complete recognition variable FLAG2 is 1, it is 
indicated that the DMA transfer of one of the video data to 
the picture composing memory 6 is presently being made. 
On the other hand, when the value of the video signal read 
complete recognition variable FLAG2 is 0, it is indicated 
that no DMA transfer of the video data is presently being 
made. When the DMA transfer from the picture composing 
memory 6 to the encoding memory 7 is made during the 
DMA transfer of the video data to the picture composing 
memory 6, the video signal presently being read into the 
encoding memory 7 becomes cut at an intermediate point. 
Hence, the interrupt mask is carried out until the value of the 
video signal read complete recognition variable FLAG2 
becomes 0. 

When the value of the video signal read complete recog- 
nition variable FLAG2 becomes 0, the DMA transfer is 
made from the picture composing memory 6 to the encoding 
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memory 7. When this DMA transfer is completed, the 
interrupt mask is cleared, and the encoding process and/or 
the transmission process are carried out, so as to repeat the 
series of operations described above. In this embodiment, 
the reading of the video signal continues while the encoding 
process and/or the transmission process are being carried 
out. For this reason, it is possible to read an amount of video 
signal that can be read, without deteriorating the encoding 
efficiency. Hence, the picture composing process can be 
carried out efficiently. 

FIGS. 17Aand 17B respectively are diagrams for explain- 
ing reading frames. More particularly, FIG. 17A shows the 
reading frames in the prior art, and FIG. 17B shows the 
reading frames in this embodiment. In FIGS. 17A and 17B, 
the video signals from the 4 video signal sources 1-1 through 
1-4 are respectively denoted by #1 through #4, 

FIG. 17A shows a case where the synchronized video 
signals such as those shown in FIG. 1 are composed. In this 
case, 1 frame is divided into 4, the video signals from the 4 
video signal sources 201 are denoted by #1 through #4 and 
composed, and the composed video signal is encoded and 
transmitted from the transmission unit 203. When transmit- 
ting the composed video signal via the network 204 which 
has a narrow transmission band, the compression and encod- 
ing are carried out. But when the compression and encoding 
are carried out, some frames are discarded due to the time 
required to carry out the encoding process, the amount of 
information generated and the like. Accordingly, the frames 
which are compressed and transmitted are actually thinned, 
and the frame rate is low. 

On the other hand, FIG. 17B shows this embodiment 
wherein the asynchronous video signals #1 through #4 from 
the 4 video signal sources 1-1 through 1-4 are successively 
read, the video signals #1 through #4 are composed and 
subjected to the encoding process. When the composed 
video signal is transmitted via the network 214 which has a 
narrow transmission band, the thinning occurs and a high 
frame rate cannot be obtained, as described above. Hence, in 
this embodiment, after reading the video signal #1 amount- 
ing to 1 frame, for example, the next video signal #2 
amounting to 1 frame is read, and the video signals #3, #4, 
#1, #2, . . . are successively read in units of 1 frame. In other 
words, the frames are essentially thinned before the pictures 
are composed, and the composed picture is subjected to the 
encoding process. 

FIGS. 18A and 18B respectively are diagrams for explain- 
ing encoding frames. 

FIG. 18A shows the encoding frames in the prior art 
corresponding to FIG. 17A, with respect to the frames (1), 
(4), (6) and (7) which are actually compressed and trans- 
mitted in the case shown in FIG. 17A. As shown in FIG. 
18A, 1 frame is formed by composing the video signals #1 
through #4 and the encoding process is carried out in the 
prior art. Hence, contents of each of the frames (1), (4), (6) 
and (7) are updated. 

On the other hand, FIG. 18B shows the encoding frames 
in this embodiment corresponding to FIG. 17B. As shown in 
FIG. 18B, the video signal #1 is read, composed and 
encoded in the frame (1), and only the portion of the video 
signal #1 is updated. In the next frame (4), the video signals 
#2 through #4 are read, composed and encoded, and the 
portion of the video signal #1 is not updated while the 
portions of the video signals #2 through #4 are updated. 

In the next frame (6), the video signals "1 and #2 are read, 
composed and encoded, and the portions of the video signals 
#1 and #2 are updated while the portions of the video signals 
#3 and #4 are not updated. In the next frame (7), the video 
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signal #3 is read, composed and encoded, and only the 
portion of the video signal #3 is updated. Accordingly, the 
difference between the frames is zero for the portion which 
is not updated, and the difference between the frames is only 
generated for the portion which is updated. As a result, the 
encoding process is simple compared to the case where the 
entire 1 frame of the composed picture is subjected to the 
encoding process. 

FIGS. 19Aand 19B respectively are diagrams for explain- 
ing receiving and displaying frames. 

FIG. 19A shows the receiving and displaying frames for 
the prior art corresponding to FIGS. 17 A and 18 A In the 
case of the prior art shown in FIG. 19A, the frames (1), (4), 
(6) and (7) are received, and during the time corresponding 
to the frames (2) and (3) the contents obtained by a decoding 
process carried out with respect to the preceding frame (1) 
are repeatedly displayed. In addition, during the time cor- 
responding to the frame (5), the contents obtained by the 
decoding process carried out with respect to the preceding 
frame (4) are displayed. 

On the other hand, FIG. 19B shows the receiving and 
displaying frames in this embodiment corresponding to 
FIGS. 17B and 18B. For example, with respect to the video 
signal #1, the contents of the frame (1) are repeatedly 
displayed up to the frame (6) which is updated. In addition, 
with respect to the video signal #2, the contents of the frame 
(1) is repeatedly displayed up to the frame (4) which is 
updated. 

For example, when carrying out the picture composing 
process by CCIR601 for an effective area of 704 pixelsx480 
lines, the amount of data per 1 pixel is 1 byte, and the line 
memory 5 shown in FIG. 5 must store 704 bytes of data per 
1 line. Hence, in order to provide a sufficient margin for 
rewriting data, it is desirable that the line memory 5 has a 
memory capacity of approximately 2 kbytes. In addition, it 
is desirable tat the picture composing memory 6 shown in 
FIG. 5 has a memory capacity of approximately 704x480x 
1.5 bytes, that is, approximately 512 kbytes, for example, by 
taking into consideration the color difference signal and the 
like. The memory capacity of the picture composing 
memory 6 is independent of the number of pictures to be 
composed. 

FIG. 20 is a system block diagram showing a fourth 
embodiment of the multiple picture composing apparatus 
according to the present invention. In FIG. 20, those parts 
which are the same as those corresponding parts in FIG. 5 
are designated by the same reference numerals, and a 
description thereof will be omitted. 

The multiple picture composing apparatus shown in FIG. 
20 includes, in addition to those parts shown in FIG. 5, a 
video signal source input judging unit 21 including video 
signal source processing parts 22-1 through 22-n, a 
horizontal/vertical direction reducer 23, a decoding memory 
24, a decoder 25, a decoding controller 26, and a special data 
storing memory 27 which are coupled as shown. 

In this embodiment, the video signal source input judging 
unit 21 judges whether or not the video signals output from 
the plurality of video signal sources (television cameras) 1-1 
through 1-n arc input thereto. If the video signal is not output 
from a certain video signal source due to a failure, no power, 
connection failure or the like of this certain video signal 
source, measures are taken so as not to compose the picture 
corresponding to this certain video signal source. As a result, 
it is possible to eliminate unstable conditions of the system 
that would otherwise occur if an unstable video signal were 
mixed, and the efficiency of the picture composing process 
is improved. 
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Each of the video signal source processing parts 22-1 
through 22-n of the video signal source input judging unit 21 
judges whether or not the video signal is output from a 
corresponding one of the video signal sources 1-1 through 
1-n and input thereto by judging whether or not a vertical 5 
synchronizing signal and a horizontal synchronizing signal 
or a color burst signal and a chrominance signal are output 
from the corresponding video signal source during a prede- 
termined time. Further, each of the video signal source 
processing parts 22-1 through 22-n can also judge whether 10 
or not the video signal is output from the corresponding 
video signal source and input thereto based on a level 
detection of the video signal. 

The decoder 25 decodes the received encoded video 
signal under the control of the decoding controller 26, and 15 
stores a decoded video signal in the decoding memory 24. 
Special data from the picture composing controller 9 is 
written into the special data storing memory 27. The 
horizontal/vertical direction reducer 23 carries out a reduc- 
ing process by thinning pixels or lines so as to obtain a 20 
specified picture size. 

FIG. 21 is a flow chart for explaining a picture composing 
process of the fourth embodiment. This picture composing 
process which includes the video input judgement for judg- 
ing whether or not the video signal input exists is also 25 
applicable to the picture composing process of each of the 
first through third embodiments described above. 

In FIG. 21, a step Gl first carries out an initializing 
process, and a step G2 substitutes 1 into A. A step G3 sets 
C A in the video signal selector 2, and a step G4 sets the 30 
odd/even field read variable 0^ . A step G5 sets the incre- 
ment value V A of the picture composing memory 6, and a 
step G6 sets the horizontal direction reduction value H A . A 
step G7 substitutes XS A , YS A into the picture transfer 
destination address XO, YO, and a step G8 substitutes 1 into 35 
the video signal read complete recognition variable FLAG. 
A step G9 sets the interrupt generation condition VS A , and 
a step G10 clears the interrupt mask. 

A step Gil decides whether or not FLAG-0, and a step 
G12 masks the interrupt if the decision result in the step Gil 40 
is YES. A step G13 carries out an encoding process and/or 
a transmission process. A step G14 decides whether or not 
A=n, and the process advances to a step G17 if the decision 
result in the step G14 is NO. The step G17 increments A by 
1, and substitutes 1 into A if A>n. 45 

On the other hand, if the decision result in the step G14 
is YES, a step G15 carries out the video input judging 
process to judge whether or not the video signal input exists. 
A step G16 carries out an invalid area process if there is no 
video signal input. After the step G16, the process advances 50 
to the step G17 to increment A by 1 or, to substitute 1 into 
A if A>n. After the step G17, the process returns to the step 
G3 so as to repeat the above described process. 

FIG. 22 is a How chart for explaining the video input 
judging process of this embodiment. First, a step HI masks 55 
the interrupt, and a step H2 substitutes 0 into each of a video 
input holding variable VI and a no-video input holding 
variable VD. A step H3 substitutes 0 into an input judging 
check variable D. A step H4 checks a video signal input from 
a video signal source C N . Dy and a step H5 decides whether 60 
or not the video signal input exists. If the decision result in 
the step H5 is YES, a step H6 increments VI by 1. On the 
other hand, a step H7 switches C N _ V and C N _ vn and incre- 
ments VD by 1 if the decision result in the step H5 is NO. 

After the step H6 or H7, a step H8 increments D by 1, and 65 
a step H9 decides whether or not D=N. The process returns 
to the step H4 if the decision result in the step H9 is NO. If 



,335 Bl 

20 

the decision result in the step H9 is YES, a step H10 
substitutes VI into n, and a step HU carries out an initial- 
izing process. In other words, the picture composing process 
can be carried out by setting the value of the video input 
holding variable VI as the number n of video signal sources. 

For example, if C>1-1, C 2 -l-2, C 3 -l-3, C 4 -l-4, n-4, 
and N-4 before the video input judging process in the case 
where the video signal sources 1-1 through 1-4 exist, it 
becomes Q-l-4, C 2 -l-2, C 3 «l-1, C 4 «l-3, n=2, and N=4 
when no video signals are input to the video signal source 
input judging unit 21 from the video signal sources 1-1 and 
1-3. Hence, the picture composing process in this particular 
case composes 2 pictures using the video signals from the 
video signal sources 1-2 and 1-4. 

FIG. 23 is a flow chart for explaining the invalid area 
process of this embodiment. A block within the picture 
composing memory 6 corresponding to no video signal input 
as a result of the video input judging process described 
above becomes an invalid area of the composed picture. 
Accordingly, the invalid area process shown in FIG. 23 
inserts special data within this invalid area. 

In FIG. 23, a step Jl decides whether or not n=2 or 3 as 
a result of the video input judging process. The process ends 
if the decision result in the step Jl is NO. On the other hand, 
if the decision result in the step Jl is YES, a step J2 obtains 
an arrangement for N and data group thereof. 

Next, a step J3 writes a fixed pattern in the special data 
storing memory 27 shown in FIG. 20. A step J4 sets a 
vertical direction reduction value V^y/2, and a step J5 sets a 
horizontal direction reduction value H^. A step J6 sets a 
picture composing start address XS N , YS^. A step J7 sets a 
special data storing address, a step J8 carries out a data 
transfer by a DMA2 of the DMAC 14, and the process ends. 

In a case where N=4 and the Nth, that is, the 4th data is 
obtained from the arrangement for n~4, the following sub- 
stitutions are made. 

XS^XS+X/2 

XE^XS+X 

YSjv-YS+Y/2 

YE^YS+Y 

H^l/2 

^^iV" ^ START 

vE^y^j^ 

A horizontal and vertical direction start address of the 
special data storing memory 27 is set to XD, YD, and a write 
process is carried out with respect to this address XD, YD to 
write some kind of fixed pattern. For example, fixed color 
data such as fixed black and fixed blue, fixed bit map data 
such as manufacturer's logo, message such as "composing 
picture" and "no input from video signal sources 1 & 4", and 
other user data or the like may be used as the fixed pattern. 

After the special data is written into the special data 
storing memory 27 described above, the horizontal direction 
reduction value H^ and the vertical direction reduction value 
Vjv/2 in the horizontal/vertical direction reducer 23. In this 
case, Vjy corresponds to an increment value which becomes 
2 when reading 1 frame without the thinning and becomes 
1 when reading 1 field with the thinning. With respect to the 
DMAC 14, the start address XS^ YS^ of the picture 
composing memory 6 is specified as the transfer destination 
address, the start address XD, YD of the special data storing 
memory 27 is specified as the transfer source address, and 
the data transfer amounting to XxY is carried out. 

The data amounting to XxY which is transferred under the 
control of the DMAC14 is thinned by in both the 
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horizontal and vertical directions by the horizontal/vertical 
direction reducer 23, and stored in the Nth block, that is, the 
4th block of the picture composing memory 6. This invalid 
area process is called every time one round of the channel 
switching of the video signal sources is made, and for this 5 
reason, it is possible to actively modify the contents of the 
special data and transmit the special data to the receiving 
end. 

FIG. 24 is a flow chart for explaining another invalid area 
process of this embodiment. The invalid area process shown 10 
in FIG. 24 is applicable to a case where the received and 
decoded data is inserted into the invalid area and returned. 
In FIG. 24, steps Kl and K2 respectively are identical to the 
steps Jl and J2 shown in FIG. 23. If the decision result in the 
step Kl is YES, the step K2 is carried out, and a step K3 15 
checks the value of a decoding frame update management 
variable RCV. More particularly, the step K3 decides 
whether or not RCV«1. The decoding controller 26 changes 
the value of the decoding frame update management vari- 
able RCV to 1 every time 1 frame is decoded, and changes 20 
the value of the decoding frame update management vari- 
able RCV to 0 when a result of the decoding process is 
transferred to the picture composing memory 6. In other 
words, the decoding frame update management variable 
RCV manages whether or not the contents of the decoding 25 
memory 24 are changed after the invalid area process is 
started. Accordingly, if RCV«0 and the decision result in the 
step K3 is NO, the process ends because data identical to the 
previous data is stored in the decoding memory 24 and there 
is no need to overwrite the data. 30 

If the decision result in the step K3 is YES, steps K4 
through K6 similar to the steps J4 through K6 shown in FIG. 
23 are carried out. Hence, the vertical direction reduction 
value Vjy/2, the horizontal direction reduction value H^, and 
the picture composing start address XS^, YSjy are set. A step 35 
K7 sets a decoded received data storing address in place of 
the special data storing address. A step K8 transfers the data 
under the control of the DMAC 14, and a step K9 substitutes 
0 into the decoding frame update management variable RCV 
upon completion of this data transfer. In this case, the 40 
received and decoded data is reduced by the horizontal/ 
vertical reducer 23, and written into the block of the picture 
composing memory 6 corresponding to the invalid area. In 
addition, this received and decoded data is composed with 
the video signals from the video signal sources and trans- 45 
mitted. 

In this case, the invalid area process is called every time 
one round of the channel switching of the video signal 
sources is made, and for this reason, it is possible to 
constantly update and transmit the received and decoded 50 
data. Accordingly, the transmitted picture is returned to the 
transmitting end, thereby making it possible to confirm the 
transmitted picture at the transmitting end. In addition, in the 
case of a system which can carry out the decoding process 
with respect to encoded recorded data presto red in another 55 
memory, it is possible to easily compose the present or live 
picture and the recorded picture which is reproduced so to 
obtain a composed picture. In this case, it is possible to 
easily obtain a composed picture such as that shown in FIG. 
2C described above, for example. 60 

FIG. 25 is a flow chart for explaining still another invalid 
area process of this embodiment. In this still another invalid 
area process, the invalid area within the composed picture is 
utilized so as to enable the enlarged display of an important 
picture. In the following description, the channel number of 65 
the video signal source which picks up the important picture 
is weighted by F 3 . 
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In FIG. 25, a step LI first decides whether or not n«2. If 
the decision result in the step LI is NO, a step L4 decides 
whether or not n«3. If the decision result in the step L4 is 
NO, n is not 2 or 3, and thus, the process ends. 

When n=2, a picture composing process using blocks bl 
and b2 of the picture composing memory 6 is carried out as 
shown in FIG. 26A, for example. In this case, the blocks bl 
and b2 respectively have a size corresponding to a picture 
read region Y/2 excluding Y/4 on top and bottom in FIG. 
26C. In addition, by making the size of the blocks bl and b2 
the same, it becomes unnecessary to take into consideration 
the weighted channel number. 

Accordingly, if the decision result in the step LI is YES, 
a step L2 obtains an enlarged composing arrangement from 
a first memory block of the picture composing memory 6, 
and a step L3 obtains an enlarged composing arrangement 
from a second memory block of the picture composing 
memory 6. These enlarged composing arrangements are set 
in advance depending on the video signal sources or the like. 
In this case, XS A , XE A , YS A , YE A , H A , V A , 0 A , VS A and VE A 
are obtained from the arrangements which are obtained for 
A=l and A=2, for example. More particularly, the following 
are obtained, for example. 

XS^ s XS 2 ^XS 

XEj-XE^XS+X 

YSj-YS 

YS 2 =YS+Y/2 

YEi-XS+Y/2 

YE 2 oYS+Y 

V„ V 2 o 2 
0 2 , 0 2 »1 

VS,, VS 2 -V 5ZAi?r flY/4 

VEj^ ^^Z^^START^^^ 

The odd field and the even field of the video signal are 
read and are alternately stored in the picture composing 
memory 6. Hence, the increment value V A of the picture 
composing memory 6 becomes 2, and the odd/even field 
read variable 0 A becomes 1. 

When n-3, the picture composing process is carried out as 
shown in FIG. 26B, for example. In this case, the channel of 
the video signal stored in the block b2 is weighted by F a . In 
other words, a step L5 substitutes 1 into D, and a step L6 
decides whether or not C^-Fj. If the decision result in the 
step L6 is NO, a step L7 increments D by 1, and a step L8 
decides whether or not D>n, The process returns to the step 
L6 if the decision result in the step L8 is NO, On the other 
hand, if the decision result in the step L8 is YES, no 
weighted video signal channel exists for D»l to D=n, and 
thus, the process ends. Further, if the decision result in the 
step L6 is YES, a step L9 switches the video signal sources 
C 2 and C 0 , and a step L10 obtains the enlarged composing 
arrangement from the second memory block of the picture 
composing memory 6. 

In other words, C^ and F a are compared for D=l to D=n, 
and the video signal source Q> and the video signal source 
C 2 corresponding to the block b2 are switched when Q^Fj. 
Hence, the video signal is stored by allocating the channel 
F 2 to the block b2 shown in FIG. 26B. Similarly to the case 
where n=2, a step L10 obtains the enlarged composing 
arrangement from the second memory block of the picture 
composing memory 6. As a result, XS A , XE A , YS A , YE A , H A , 
V A , 0 A , VS A and VE A are obtained from the arrangement 
obtained for A-2. The particular setting in this case may be 
similar to that for the case where n-2. 
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The reading of the video signal from the block b2 is made if the zoom-up picture and the picture which is not zoomed 

for both the odd and even fields, and the odd and even fields up are displayed adjacent to each other, 

are alternately stored in the picture composing memory 6. The zoom-up reading arrangement and set value group 

Accordingly, the increment value V A of the picture compos- may be set in advance depending on the system. In this case, 

ing memory 6 becomes 2, and the odd/even field read 5 XS A , XE A , YS A , YE A , H A , V A , 0 A , VS A and VE A are 

variable 0 A becomes 1. In addition, an interrupt signal is obtained from the arrangement which is obtained for A=N. 

generated at lines (YS+Y/4) and (YS+3Y/4) of the video More particularly, the following are obtained, for example, 
signal, and DMA transfer start and end processes are carried 

out. With respect to the blocks bl and b2 shown in FIG. 26B, XS^XS+X/2 

only the odd fields are read, for example, and the read odd 10 XE^XS+X 

fields are consecutively stored in the picture composing YS^YS+Y/2 

memory 6, similarly as in each of the embodiments YE^YS+Y 

described above wherein 4 pictures are composed to obtain H^l 

the composed picture. Vy*2 

FIGS. 27 and 28 are flow charts for explaining an invalid 35 0^1 

area process including a zoom-up process in this embodi- VS^-V start 

ment. A channel number which is weighted in order to make VE^V^^ 
a zoom-up display in the invalid area within the composed 

picture is denoted by F 2 . A step Ml shown in FIG. 27 As described above, both the odd and even fields are read, 

decides whether or not n=2 or 3, similarly to the step Jl 20 and the odd and even fields are alternately stored in the 

shown in FIG. 23 and the step Kl shown in FIG. 24. The picture composing memory 6. Accordingly, the increment 

process ends if the decision result in the step Ml is NO. value V^. becomes 2, and the odd/even field read variable 

If the decision result in the step Ml is YES, a step M2 becomes 1. 

substitutes 1 into D, and a step M3 decides whether or not After the above described process is completed, a video 

C Z> «F 2 . If the decision result in the step M3 is NO, a step M4 25 signal read process for reading the video signals from the 

increments D by 1, and a step MS decides whether or not video signal sources is started. First, A is set to A=N, and the 

D>n. The process returns to the step M3 if the decision result special data storing memory 27 shown in FIG. 20 is sued. In 

in the step M5 is NO. On the other hand, the process ends addition, C A is set in the video signal selector 2. If the value 

if the decision result in the step M5 is YES. of C A is F 2) the video signal selector 2 selects the video 

If the decision result in the step M3 is YES, a step M6 30 signal from the video signal source F 2 . In addition, the 

switches the video signal sources C 2 and C n . A step M7 picture composing increment value is set in the DMAC 

obtains an enlarged composing arrangement from a Nth 13, and the horizontal direction reduction value is set in 

memory block of the picture composing memory 6, and a the horizontal direction reducer 4. At the same time, the read 

step M8 substitutes N into A. A step M9 sets C A in the video position XD, YD within the special data storing memory 27 

signal selector 2 shown in FIG. 20, and a step M10 sets the 35 is substituted into the picture transfer destination address 

increment value V A of the picture composing memory 6. A XO, YO. That is, the video signal from the video signal 

step Mil sets the horizontal direction reduction value H A , source is temporarily stored in the special data storing 

and a step M12 substitutes XD, YD into the picture transfer memory 27, not the picture composing memory 6. 

destination address XO, YO. A step M13 shown in FIG. 28 When the above described process is completed, 1 is 

substitutes 1 into the video signal read complete recognition 40 substituted into the video signal read complete recognition 

variable FLAG, and a step M14 sets the interrupt generation variable FLAG, VS^ is set as the interrupt generation 

condition VS A . A step M15 clears the interrupt mask. condition, and the interrupt mask is cleared. As a result, by 

A step M16 decides whether or not FLAG^O. If the a process carried out within the interrupt handler 11, the data 

decision result in the step M16 is YES, a step M17 masks the amounting to 1 frame not subjected to the thinning process 

interrupt, and a step M18 sets the vertical direction reduction 45 is read into the special data storing memory 27 by a video 

value V A /2, and a step Ml 9 sets the horizontal direction signal read means similar to that for the case where n-1, and 

reduction value ll A . A step M20 sets a picture composing the video signal read complete recognition variable FLAG 

start address XS A , YS A , and a step M21 sets a data transfer consequently becomes 0. 

destination address XD+XZ, YD+YZ. A step M22 carries While the interrupt handler 11 is carrying out the process, 

out a data transfer by a DMA2 of the DMAC 14. 50 the picture composing controller 9 monitors the value of the 

The weighted channel F 2 which indicates the zoom-up video signal read complete recognition variable FLAG, and 

display regards a region from an address (XZ, YZ) to an waits until the value becomes 0. When the value of the video 

address (XZ+X/2, YZ+Y/2) as the picture reading region as signal read complete recognition variable FLAG becomes 0, 

shown in FIG. 26D, for example, and carries out the DMA the reading of the video signal from the video signal source 

transfer so as to read the data into the picture composing 55 amounting to 1 frame is completed. Hence, the interrupt is 

memory 6 at a magnification 1. In other words, since the masked, and with respect to the DMAC 14, the start address 

thinned video signals are composed for the other blocks, the XS A , YS A of the picture composing memory 6 is specified as 

video signal which is composed in the invalid area appears the transfer destination address, and the start address 

enlarged. XD+XZ, YD+YZ of the special data storing memory 27 is 

The switching of the video signal sources C D and C„ is 60 specified as the transfer source address, so as to carry out the 

carried out in the step M6 for the following 2 reasons. First, data transfer amounting to XxY. The data amounting to XxY 

when the invalid area process is called, the video signal transferred by the DMAC 14 is stored in the Nth block 

channel of the video signal source is read into the block n within the picture composing memory 6, without being 

within the picture composing memory 6, and the above thinned in the horizontal and vertical directions, that is, with 

described switching is made so as to reduce the lime loss 65 a magnification 1. By the above described process, the 

caused by the switching of the channels. Second, it is more received and displayed picture becomes an enlarged picture 

convenient for the user from the point of view of observation as shown on the top right of FIG. 2D, for example. 
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FIG. 29 is a flow char for explaining a picture composing 
process of a fifth embodiment of the multiple picture com- 
posing apparatus according to the present invention. In this 
embodiment, a video signal of a weighted channel is read 
and process with priority over video signals of other chan- 
nels. The picture composing process can be summarized by 
the following processes which include: (a) carrying out an 
encoding process and/or a transmission process after a read 
process is completed with respect to all video signals, (b) 
carrying out a read process to read a video signal that can be 
read during the encoding process, and (c) carrying out an 
encoding process and/or a transmission process every time 
the reading of the video signal from 1 video signal source is 
completed. Accordingly, the picture which is received and 
displayed becomes a composed picture which is updated in 
units of blocks. 

For example, in a system which encodes and transmits 30 
frames per second, the frame rate per block becomes 
approximately 3 to 7.5 when 4 pictures are composed. In 
such a system, it may be desirable to make the movements 
in a picture from a certain video signal source more clearly 
and easily visible to the user. Hence, this embodiment 
carries out a weighting with respect to such certain video 
signal source, and the video signal from this certain video 
signal source is read with priority over others, so that the 
frame rate is increased and the movements in the received 
and displayed picture of the video signal from this certain 
video signal source are more clearly and easily visible. 

A description will be given of a case where the weighting 
described above is carried out with respect to a channel 
number F 3 . In FIG. 29, a step Nl carries out an initializing 
process, and a step N2 substitutes 1 into variables A and B. 
A step N3 substitutes 1 into D, and a step N4 decides 
whether or not Q>=F 3 . If the decision result in the step N4 
is NO, a step N5 increments D by 1, and a step N6 decides 
whether or not D>n. The process returns to the step N4 if the 
decision result in the step N6 is NO. On the other hand, the 
process advances to a step N8 if the decision result in the 
step N6 is YES. 

If the decision result in the step N4 is YES, a step N7 
switches the video signal sources C x and C^. In other words, 
C D and F 3 are compared for D«l to D-n, and C D and C x are 
switched if and F 3 are the same. By this switching of 
and Cj, a video signal of the same channel as F 3 is stored in 
a block 1 of the composed picture corresponding to a 
channel number 1, and a video signal of the channel number 
1 is stored in a block of the composed picture corresponding 
to the channel number F 3 . 

Next, similarly as in the case of a set process in each of 
the embodiments described above, a step N8 sets C A , a step 
N9 sets V A , and a step N10 sets H A . A step Nil substitutes 
XS A , YS A into the picture transfer destination address XO, 
YO, and a step N12 substitutes 1 into the video signal read 
complete recognition variable FLAG. A step N13 sets VS A , 
and a step N14 clears the interrupt mask. A step N15 decides 
whether or not FLAG-0. If the decision result in the step 
N15 is YES, a step N16 masks the interrupt, and a step N17 
carries out an encoding process and/or a transmission pro- 
cess. 

A step N18 decides whether or not A=l, If the decision 
result in the step N18 is NO, a step N19 substitutes 1 into A, 
and the process returns to the step N8. On the other hand, if 
the decision result in the step N18 is YES, a step N19 
increments B by 1, substitutes 2 into B if B>n, and substi- 
tutes the value of B into A. For example, in a case where the 
video signal of the channel C x (=F 3 ) is read and the step N18 
decides whether or not A-»l, the process advances to the step 
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N19 because A and B are initially A-l and B-l. In this case, 
B is incremented by 1 and this B is substituted into A in the 
step N19 so that A and B become A=B«2. Then, the 
variables are set again, the video signal of the channel C 2 is 

5 read, and the step N18 again decides whether or not A=»l. 
This time, the decision result in the step N18 becomes NO, 
and the step N20 substitutes 1 into A. Further, after setting 
the variables corresponding to A=l the video signal of the 
channel Q (=F 3 ) is read. 

10 Accordingly, in the case where 4 pictures are composed, 
the video signal of the channel C a (=F 3 ) which is weighted 
is read once every two times, that is, read in a manner 
Cj— *C 2 — *C 1 -»C 3 -*C a -»C 4 -» , C 1 . . . and composed. Hence, 
it is possible to increase the frame rate of this channel C r 

is For example, the frame rate can be set to 7 to 15 frames per 
second in a case corresponding to the particular case 
described above. 

As described above, the picture composing process and 
the encoding process are directly related. For example, it is 

20 possible to set quantization steps for every block of the 
composed picture, and improve the resolution of an impor- 
tant block. In addition, it is possible to set a search range for 
searching a motion vector for every block, and improve the 
picture quality of the displayed picture. 

25 FIG. 30 is a diagram for explaining CIF (Common 
Intermediate Format). In a H.261 encoding process, the 
picture data is treated as having a size according to the CIF 
or the QCIF (Quarter Common Intermediate Format). In the 
case of the CIF, 1 frame includes 352x288 pixels as shown 

30 in FIG. 30. This CIF frame is made up of 12 blocks GOB, 
and each block GOB is made up of 33 macro blocks as 
shown on an enlarged scale in the lower portion of FIG. 30. 
In addition, each macro block is made up of 4 luminance 
blocks Y, and 2 color difference blocks U and V. The 

35 luminance block Y is made up of 4 blocks. The encoding of 
the CIF frame is basically made in units of macro blocks. 

FIG. 31 is a system block diagram for explaining an 
encoder part of the fifth embodiment. FIG. 31 shoes a case 
where the encoder 8 shown in FIGS. 5 and 20 has a hybrid 

40 encoding structure. In FIG. 31, those parts which are the 
same as those corresponding parts in FIGS. 5 and 20 are 
designated by the same reference numerals, and a descrip- 
tion thereof will be omitted. 

In addition to the elements described above in conjunction 

45 with FIGS. 5 and 20, the encoder 8 includes a picture 
composing apparatus 31, a difference processor 32, selectors 
33 and 34, a discrete cosine transform (DCT) unit 35, a 
quantization unit 36, a transmission buffer 37, an inverse 
quantization unit 38, an inverse DCT (IDCT) unit 39, an 

50 adding processor 40, a motion estimation unit 41, an in-loop 
filter 42, a transmission/reception processor 51, a reception 
buffer 52, an inverse quantization unit 53, an IDCT unit 54, 
an adding processor 55, an in-loop filter 56, a motion 
estimation unit 57, and a selector 58 which are coupled as 

55 shown in FIG. 31. 

The picture composing apparatus 31 includes the picture 
composing memory 6 shown in FIG. 5 or 20, and has a 
function of composing the video signals from the plurality of 
video signal sources 1-1 through 1-n. 

60 The picture composing apparatus 31 transfers the video 
signal related to the composed picture to the encoding 
memory 7, and carries out an encoding process in units of 
blocks made up of 8x8 pixels, for example, with respect to 
the video signal amounting to 1 composed picture and stored 

65 in the encoding memory 7. 

When carrying out this encoding process, an inter-frame 
prediction coding is made by controlling the selectors 33 and 
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34 to the connection states shown in FIG. 31, and a block in 
this case is referred to as an inter-block. In addition, when 
the selectors 33 and 34 are switched from the connection 
states shown in FIG. 31, an intra-frame encoding is made, 
and a block in this case is referred to as an intra-block. 

The 8x8 pixel block input to the DCTunit via the selector 
33 is subjected to a DCT process, and DCT coefficients are 
input to the quantization unit 36. In this case, due to the 
characteristic of the video signal, a low-frequency compo- 
nent including a D.C. component forms a large part of the 
DCT coefficients, and it is thus possible to eliminate a 
high-frequency component from the DCT coefficients. 
Hence, the amount of information can be compressed also 
by this DCT process. Furthermore, the DCT coefficients are 
quantized in the quantization unit 36 and transferred to the 
transfer buffer 37. In this case, when a quantization value, 
that is, a quantization step, of the quantization unit 36 is set 
small, the amount of information generated becomes large 
but the picture quality of the received and displayed picture 
becomes satisfactory. On the other hand, when the quanti- 
zation value is set large, the amount of information gener- 
ated becomes small, but the picture quality of the received 
and displayed picture deteriorates. 

The transmission/reception processor 51 carries out a 
transmission process to transmit the data stored in the 
transmission buffer 37 at a constant rate according to a 
transmission band of a network 214 or the like. The encod- 
ing controller 10 monitors the amount of data stored in the 
transmission buffer 37, and controls the quantization value, 
that is, the quantization step, of the quantization unit 36 so 
that no overflow or underflow occurs. 

Using the structure including the inverse quantization unit 
38, the I DCT unit 39, the adding processor 40, the motion 
estimation unit 41, the in-loop filter 42 and the like, the 
encoded frame is reproduced and an inter-frame difference 
between the frames is obtained in the difference processor 
32. The motion estimation unit 41 obtains a position where 
a difference between a previous frame and a present frame 
becomes a minimum with respect to a block having a certain 
size, describes a moving destination by a motion vector, and 
carries out a process so that the inter- frame difference 
becomes small. The in-loop filter 42 reduces the deteriora- 
tion of the inter-frame prediction efficiency caused by the 
accumulation of the distortions generated by the quantiza- 
tion. In other words, the in-loop filter 42 is a spatial lowpass 
filter which carries out a filtering process in units of blocks. 

The data received via the transmission/reception proces- 
sor 51 is input to the IDCT unit 53 via the reception buffer 
52. The IDCT unit 53 subjects the received data to an IDCT 
process. Using the structure including the adding processor 
55, the in-loop filter 56, the motion estimation unit 57 and 
the selector 58, an intra-frame decoding or an inter-frame 
decoding is carried out, and a resulting decoded data is 
stored in the decoding memory 24. 

In the quantization unit 36 described above, 1 quantiza- 
tion value can be allocated per 1 macro block in the case of 
the H.261 encoding. This quantization value can take a value 
from 1 to 31. As described above, the amount of information 
generated becomes large when this quantization value is 
small, and the amount of information generated becomes 
small when this quantization value is large. In other words, 
if the quantization value is generally small, it is possible to 
transmit a picture having a satisfactory picture quality and in 
which movements are smooth. On the other hand, if the 
quantization value is large, the picture quality is poor and the 
movements are not smooth and appear intermittent. 

The data subjected to the series of encoding processes is 
stored in the transmission buffer 37 at a rate depending on 



6,335 Bl 

28 

the amount of information generated by the encoding pro- 
cess. The data stored in the transmission buffer 37 are 
transmitted by the transmission/reception processor 51 at a 
constant rate corresponding to the transmission band of the 

5 network 214. In order to prevent the overflow of the trans- 
mission buffer 37, the quantization value is increased when 
the stored amount of data in the transmission buffer 37 
increases and exceeds a predetermined value. On the other 
hand, in order to prevent the underflow of the transmission 

10 buffer 37, the quantization value is decreased when the 
stored amount of data in the transmission buffer 37 decreases 
and becomes the predetermined value or less. The quanti- 
zation value is controlled in this manner by controlling the 
quantization unit 36 by the encoding controller 10. 

is It is assumed for the sake of convenience that among the 
usable quantization values, the smallest quantization value is 
denoted by Qmin, the largest quantization value is denoted 
by Qmax, and the quantization value which is actually 
selected is denoted by Q. For example, Qmin and Qmax are 

20 set to Qmin=3 and Qmax=12, and if the memory capacity of 
the transmission buffer 27 is 1 kbytes, the relationship of the 
occupied amount of the transmission buffer 27 and the 
quantization value can be controlled as follows. 
Q=12 if (occupying amount) ^900bytes. 

25 Q=ll if 800 bytes^(occupying amount)<900 bytes. 
Q-10 if 700 bytes ^(occupying amount)<800 bytes. 
Q=9 if 600 bytes ^(occupying amount)<700 bytes. 
Q=8 if 500 bytes ^(occupying amount)<600 bytes. 

30 Q=7 if 400 bytes ^(occupying amount)<500 bytes. 
Q=6 if 300 bytes ^(occupying amount)<400 bytes. 
Q=5 if 200 bytes ^(occupying amount)<300 bytes. 
Q-4 if 100 bytes ^(occupying amount)<200 bytes. 

35 Q=3 if (occupying amount)<100 bytes. 

The quantization value in the quantization unit 36 is 
controlled with respect to the weighted channel differently 
from other channels, so that the picture quality of the 
received and displayed picture is satisfactory for the 

40 weighted channel. 

FIG. 32 is a flow chart for explaining a quantization value 
determining process of the fifth embodiment. In this case, a 
weighted channel is denoted by F 4 . First, a step PI shown in 
FIG. 32 substitutes 1 into D, and a step P2 decides whether 

45 or not C £) »F 4 . If the decision result in the step P2 is NO, a 
step P5 increments D by 1, and a step P6 decides whether or 
not D>n. The process returns to the step P2 if the decision 
result in the step P6 is NO. On the other hand, if the decision 
result in the step P6 is YES, a step P9 sets the minimum 

50 quantization value Qmin and the maximum quantization 
value Qmax as the usable quantization range. 

If the decision result in the step P2 is YES, a step P3 
obtains D area information within the picture composing 
memory 5, and a step P4 decides whether or not an encoding 

55 processing is being made with respect to the inside of the D 
area. If the decision result in the step P4 is YES, a step P7 
sets a minimum quantization value Qmin-a and a maximum 
quantization value Qmax-ct as the usable quantization 
range. On the other hand, if the decision result in the step P4 

60 is NO, a step P8 sets a minimum quantization value Qmin+p 
and a maximum quantization value Qmax+p as the usable 
quantization range. Both a and (3 take values of approxi- 
mately 1 to 5. After the step P7, P8 or P9, a step P10 
determines the quantization value within the usable quanti- 

65 zation range depending on the occupied amount of the 
transmission buffer 37. In addition, a step pll carries out the 
quantization, and the process ends. 
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For example, if the minimum quantization value Qmin is 
3, the maximum quantization value Qmax is 12, and both a 
and p are 2, the minimum quantization value Qmin and the 
maximum quantization value Qmax with respect to the 
video signal of the weighted channel respectively become 1 
and 10, while the minimum quantization value Qmin and the 
maximum quantization value Qmax with respect to the 
video signals of the other channels respectively become 5 
and 15, and the amount of information generated for the 
composed picture as a whole becomes approximately the 
same as the amount of information generated when no 
quantization value control with priority is carried out. 

When composing 4 pictures, if the CIF blocks 1 to 12 
shown in FIG. 30 are respectively denoted by GOBI to 
GOB12, GOBI, GOB3 and GOB5 are set as the areas in 
which the quantization value is decreased in the usable 
quantization range in the step P7 when D=l, Similarly, when 
D-2, GOB7, GOB9 and GOB11 are set as the areas in which 
the quantization value is decreased in the usable quantiza- 
tion range in the step P7. In addition, GOB2, GOB4 and 
GOB6 are set as the areas in which the quantization value is 
decreased in the usable quantization range in the step P7 
when D=3, and GOB8, GOB10 and GOB12 are set as the 
areas in which the quantization value is decreased in the 
usable quantization range in the step P7 when D«4. 
Accordingly, by setting the amount of information generated 
as a whole to a constant value and decreasing the quantiza- 
tion value of only the video signal of the weighted channel, 
it is possible to transmit a picture having a satisfactory 
picture quality. 

In the embodiment described above, the quantization 
value control with priority is carried out with respect to the 
weighted channel. However, it is possible to similarly con- 
trol whether or not to carry out an inter-frame prediction 
coding. In other words, the distortion caused by the encod- 
ing is small in the case of the intra-frame encoding, but the 
amount of information generated is large. On the other hand, 
in the case of the inter-frame prediction coding, the distor- 
tion caused by the encoding is large and the amount of 
information generated is small because the encoding is made 
using the difference between the areas within the previous 
and present frames. 

Accordingly, the encoding process may be switched so 
that the intra-frame encoding process is carried out with 
respect to the video signal of the channel which is weighted 
similarly as in the case of the weighted channel number F,, 
for the quantization value control with priority described 
above, and the inter-frame prediction coding process is 
carried out with respect to the video signal of the other 
channels. In other words, similarly to the case of the 
quantization value control with priority described above, the 
D area information within the picture composing memory 6 
is obtained, and the encoding process is switched so that the 
intra-frame encoding process is carried out with respect to 
the D area, that is, the block corresponding to the weighted 
channel, and the inter- frame prediction coding process is 
carried out with respect to the other blocks. By carry out 
such a process, it is possible to carry out the prevision coding 
control with priority to maintain the satisfactory resolution 
of the important picture. 

On the other hand, when carrying out the motion estima- 
tion process in a search range, it is possible to control the 
search range to become larger with respect to the video 
signal of the weighted channel. For example, the H.261 
motion estimation process is carried out with respect to only 
the luminance signal in units of macro blocks, and in a case 
where the search range is large, the distortion caused by the 
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encoding process is small but the search time is long because 
the possibility of finding a block with a value close to the 
present block is high, and the time required for the encoding 
process as a whole is long. On the other hand, in a case 

5 where the search range is small, the distortion caused by the 
encoding process is large but the search time is short because 
the possibility of finding a block with a value close to the 
present block is low, and the time required for the encoding 
process as a whole does not become long. 

10 Hence, if the D area information within the picture 
composing memory 6 is obtained and the encoding process 
is made with respect to the inside of the D area, that is, the 
block corresponding to the weighted channel, the motion 
estimation search range control with priority'is'carried out 

15 by setting the search range is set to ±a with respect to this 
block corresponding to the weighted channel, and the search 
range is set to ±p with respect to the other blocks. For 
example, a is 15, and p is 7 or 0 to indicate that no search 
is made. As a result, the picture can be transmitted so that the 

20 distortion caused by the encoding process is small with 
respect to the video signal of the weighted channel. 

The in-loop filter 42 shown in FIG. 31 is a lowpass filter 
which eliminates the distortion generated by the 
quantization, that is, reduces the quantization error accumu- 

25 lated in a motion estimating memory within the motion 
estimation unit 41. Accordingly, a sharp picture becomes 
softened or blurred by the in-loop filter 42. Hence, it is 
possible to carry out an in-loop filter control with priority 
and control the connection of the in-loop filter 42, so that the 

30 in-loop filter 42 is disconnected or disabled when the 
quantization value is small and the quantization error is 
small, and the in-loop filter 42 is connected or enabled when 
the quantization value is large and the quantization error is 
large. In other words, the switching of the quantization value 

35 control with priority described above and the switching of 
the in-loop filter control with priority may be controlled in 
a finked manner. 

In the case of the prediction coding control with priority 
described above, when the D area information within the 

40 picture composing memory 6 is obtained and the encoding 
process is carried out with respect to the D area, that is, the 
block corresponding to the weighted channel, the intra- 
frame encoding process is carried out and the distortion 
caused by the encoding process is small. Hence, the con- 

45 nection of the in-loop filter 42 may be controlled so as to 
disconnect the in-loop filter 42 in this case, and to connect 
the in-loop filter 42 with respect to the blocks other than the 
block in the D area since the inter-frame prediction coding 
process is carried out and the distortion caused by the 

50 encoding process is large with respect to these other blocks. 
FIG. 33 is a diagram for explaining a video signal source 
information obtaining unit. In FIG. 33, the video signals 
from the video signal sources 1-1 through 1-n are input to 
the input signal monitoring unit 3 shown in FIG. 5 or 20 via 

55 the video signal selector 2. In addition, extended data from 
the video signal sources 1-1 through 1-n are collected by an 
information obtaining unit 61 and transferred to the picture 
composing controller 9. 
The extended data include resolution information, frame 

60 rate information and the like. The video signal sources 1-1 
through 1-n respectively have a structure having a function 
of transmitting the extended data, such as a digital camera 
and a decoder of the MPEG 1, H.261, JPEG or the like. The 
resolution information includes an effective area region in 

65 the case of the digital camera, a resolution of an encoding 
frame such as 704x480 pixels in the case of the MPEG 
decoder, a resolution of a CIF frame of 352x288 pixels or a 
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QCIF frame of 176x144 pixels in the case of the H.261 carried out for every s frames, and channel of the video 

decoder, and the resolution in the case of the JPEG decoder. signal having the largest amount of motion is constantly 

On the other hand, the frame rate information includes a monitored so that the weighted channel can be switched 

frame rate of 29.97 Hz in the case of the digital camera, and cve ry time the channel of the video signal having the largest 

a decoding frame rate in the case of the MPEG1, H.261 and 5 amount of motion changes. 

JPEG decoders. i n addition, various control operations can be carried out 

The information obtaining unit 61 or the picture compos- immediately after the video signal is read in the above 

mg controller 9 determines the channel to be weighted based described manner. For example, the data is transmitted at a 

on the extended data from the video signal sources 1-1 constan , rate ^ , tQ (he Bemo± 2M and m j, ^ 

through 1-n, so that the picture composing process can be M efective tQ ^ , he enc0(Jin s , n ord6f (o 

carried out in accordance wi b each of the embodiments transmit 1Q fram{ . 1 ^ , , o a M kb 

described above, tor example, based on the frame rate u me amoun( of da(a j fram6 becom6S goo b and 

information received from the video signal sources when ^ amount of data 1 macro b , ock becomes 2 b , n 

composing the pictures, the channel having the largest frame other WQrd ., ^ desirab , e tQ ^ ^ tization value M 

rate may be weighted so that the video signal of the weighted „ r tU t t , 4 n , t , _ t u f 

/ . . . . . • , • t • , n , is that the amount of data becomes 2 bytes as a result of the 

channel is read with priority over the video signals of the encoding nrncess 

other channels. In addition, based on the resolution 1T i_ a • . a j*u j* 

, , . However, when 4 pictures are composed and the encoding 

^formation, the channel having the highest resolution may ^ ^ cM ^ fl who , e ^ t block q{ ^ 

be weighted so that the picture composing process zooms up gi j ^ fead ^ normal difference da(a fa ^ ^ 

the video signal of the weighted channel. 2Q tQ me bbck of , he newl fead video 6 si , but , he 

The weighting may be earned out using the information difference data ^ no , ted ^ tQ (he olher 

m he encoder 8 For example, in the case of a system bk)cks since ^ video si b afe ^ date(J ^ 

mc uding a hybrid encoder it is possible to utilize the , 0 me 0 , her blocks In 0(her wofd when nQ diSeKa(x data 

motion estimation process. In other words, if the mo ion ^ ated( the amount of data 1 macf0 block becomes 

vector generated by the motion estimation process that is, 25 „ b Hence, it is possible to allocate 8 bytes per 1 macro 

an address difference between an encoding block and a bbck ^ tQ , he block of ^ dated videQ gi { 

re erence block is large, it means that the difference B uti , izj me aboye describe<J ,r haracteristi % is 

between the area being processed and the previous frame ,s jb , e to control the quantization value immediately after 

large and that the video signal from the video signal source , he video si al is read Fof { ^ yariable A m (he 

includmg a large motion ts read. In the case of the video 3Q icnlre si ocess shown in nG u mdicates that 

signal mcluduig the large motion, it may be assumed that on , ^ felock A within ^ ^ composing memory 6 

some kind of an event is generated. Hence, such a video new , reads the video ^ when £ f >^ 

signal source is weighted so that the video signal from this quantization , and the differe * ce exisls omv ^ tms block A 

video signal source is read with priority over video stgnak and , he fe is zero fof the other ^ Accordingly> 

from other video signal sources or is subjected to the 3J when carr yi ng out the encoding process with respect to the 

video""? 7 wurc^s " °™ * bl ° ck A ' a 9 uanlization value which can tolerate u ? 10 4 

vi eo sign a sources. t - mes tne norma i amoun t 0 f d a t a aft er the quantization 

When sampling a moving quantity amounting to s frames fa used B controlJin lhe quantization ; alue m this 

from an encoding frame t at a certain tune a sum total V A m ft is fble (0 ca « ou ^ ^ jcture ^ 

of the motion vectors in a block A withm i the picture 4Q and , he 6nco(Ji Qcess whi £ maintain ,^ , h | 

composing memory 6 can be described by the following >moum of informatioD ated as , whole t0 fl co ^ tant 

formula, where a scalar quantity of the motion vector in the wa i ut 

horizontal direction x and the vertical direction y in the , n a case where , he inter . frame & low due t0 

encoding frame t is denoted by w , chaQge in mM Qr (he ^ (he intra . frame encoding 

45 process is selected. On the other hand, in a case where the 

u . yyy inter-frame correlation is high such as in a still picture, the 

A ~ T^thtih^ 1 ^ inter-frame prediction coding process is selected. As 

described above, the correlation of the present frame with 
respect to the previous frame is low when the video signal 

In this case, when 4 pictures are composed, the values of xj, 50 is newly read, but the correlation of the present frame with 

yj, xh and yh for the blocks 1 through 4 are as follows. respect to the previous frame is high for the other video 

signals. 

Block 1: xj-0, yj=0, xh«176, yh-144 Therefore, similarly to the case described above where the 

Block 2: xj=0, yj»144, xh«176, yh=288 quantization value is controlled, it is possible to carry out the 

Block 3: xj-176, yj«0, xh=352, yh-144 55 intra -frame encoding process with respect to the block A 

Block 4: xj=176, yj=144, xh=352, yh=288 since only the block A within the picture composing memory 

6 newly reads the video signal, and to carry out the inter- 

Thc sum total U A of the motion vectors of the block A frame prediction coding process with respect to the other 

within the picture composing memory 6 is obtained in the blocks. As a result, the prediction coding control with 

above described manner and a comparison is made for A=l 60 priority can be carried out immediately after the video signal 

to A=n. The channel C A of the video signal source 1-A is read so as to subject only the newly read frame to the 

having the largest sum total U A as a result of this comparison intra-frame coding process and to transmit the video signal 

is regarded as the channel which is to be weighted. The having a satisfactory picture quality, 

picture part which is to be enlarged or zoomed up and As described above, the motion estimation process 

extracted is specified so that a block having the largest 65 searches for a value which is within the previous frame and 

motion vector is arranged at the center. The operation of is closest to the block which is presently being encoded, 

obtaining the block having the largest motion vector is With respect to the newly read video signal during the 
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picture composing process, the possibility that a block 
which is moved from the previous frame exists is high, but 
the other blocks are the same as the previous frame and 
includes no motion. Hence, it is possible to shorten the time 
required for the encoding process by carrying out a motion 
estimation search range control with priority immediately 
after the video signal is read, so that the motion estimation 
process is carried out only with respect to the block A of the 
picture composing memory 6 and the motion estimation 
process is omitted with respect to the other blocks. 

In addition, it is possible to carry out a in -loop filter 
switching control with priority immediately after the video 
signal is read. For example, with respect to the newly read 
video signal, the inter-frame difference is large and the 
possibility of the quantization error being large is high. But 
with respect to the other video signals, the inter-frame 
difference is small, and the quantization error is also small. 
Accordingly, it is possible to carry out the in -loop filter 
switching control so that the in-loop filter 42 shown in FIG. 
31 is connected when encoding the block A of the picture 
composing memory 6, and the in-loop filter 42 is discon- 
nected when encoding the other blocks. 

FIGS. 34A through 34D respectively are diagrams for 
explaining motion estimation and arrangement of pictures of 
the composed picture. As described above, the H.261 motion 
estimation process is carried out with respect to the lumi- 
nance signal in units of macro blocks. When carrying out the 
motion estimation process with respect to a certain frame t, 
a previously encoded frame (t-1) is used. FIG. 34A shows 
a macro block with a start address (x, y) and an end address 
(x+16, y+16) of the frame t, and FIG. 34B shows a search 
range with addresses (XL, YL), (XII, YH). 

When the address of the macro block of the frame t is 
denoted by (x, y) as described above, a luminance of a pixel 
is denoted by k w , and an address of a reference macro 
block of the frame (t-1) is denoted by x', y'), a sum K^. of 
absolute differences of luminance values between the 
address (x, y) of the frame t and the address (x', y') of the 
frame (t-1) for i=0 to 15 and j-0 to 15 can be described by 
the following. Since the motion estimation process is carried 
out with respect to an area of 16x16 pixels, that is, 4 
luminance data of the macro block, (i, j) is set to 0 to 15. 

The address (x\ y') of the reference macro block can be 
selected depending on the system, and may be set within the 
entire range or within a range of ±Z with respect to the 
address (x, y). In this case, the address (x\ y') satisfies the 
following relationship, and Z may take a value of 7 or 15, for 
example. 

(XI., YL)<(x\ y*)<(XH f YH) 

where XL=0, YL=0, XH=352-16 and YH-288-16 when 
searching the entire range, and 

XL-x-Z, YL-y-Z, XH-x+Z and YH-y+Z 

when making a partial search. 

The address (x\ y') which makes the sum K^ y . of the 
absolute differences of the luminance values at the addresses 
(x, y) and (x\ y') a minimum value is regarded as the address 
of the data of the previous frame to which the reference is 
to be made. Accordingly, the motion vector in this case 
becomes x-x, y'-y). 

In addition, when composing the video signals input from 
independent video signal sources, the video signals are 
mutually independent. Hence, when carrying out the motion 
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estimation process, it may be predicted that the sum K^y, 
which takes the minimum value is selected within the same 
block of the picture composing memory 6. Thus, it is 
meaningless to carry out the motion estimation process 

5 spanning other blocks. Therefore, by limiting the address (x\ 
y') of the reference macro block described above to within 
the same block, it is possible to efficiently carry out the 
encoding process. 

For example, when a start address and an end address of 

10 each block are respectively denoted by (xl, yl) and (xh, yh) 
when composing 4 pictures, the following relationship 
stands in FIG. 34B. 

QCL t YL)<(x\ y*)<(XH, YH) 

15 where XL=xl, YL=yl, XH=xh-16 and YH=yh-16 when 
searching the entire range, and 

XL*>max(x-Z, xf), H= max (y-Z, y[), XtI-mm{x+Z t xh) and 
>7/=mm(>>+Z, yh) 

20 

when making a partial search. 

The motion estimation process is carried out only with 
respect to the following areas, and the encoding efficiency 
can be improved by limiting the search range of the motion 
25 estimation process when composing the pictures. 

Block 1: xl«0, yl-0, xh-176, yh-144 
Block 2: xl-0, yl=144, xh-176, yh-288 
Block 3: xl«176, yl=0, xh-352, yh-144 
30 Block 4: xl=176, yl«144, xh=352, yh=288 

In each of the embodiments described above, the descrip- 
tion is mainly made with respect to a case where 4 pictures 
are composed, but the number of pictures to be composed 

35 may of course be greater than 4. For example, FIG. 34C 
shows a case where 9 pictures are composed, and FIG. 34D 
shows a case where 16 pictures are composed. Generally, the 
display screen is equally divided into a predetermined 
number of parts, and the area of each part is allocated to a 

40 corresponding one of a plurality of video signal sources. 
When the number of pictures to be composed is denoted 
by N, the number of equally divided parts of the display 
screen is denoted by n, and the picture composing memory 
6 has n blocks, it is possible to equally divide the display 

45 screen as described above and compose the video signals 
amounting to N channels if N«n. If N>n, the video signals 
amounting to n channels are read in one operation, and the 
channel numbers to be read are switched for every prede- 
termined time. 

50 For example, in a case where N=6, n=4 and the prede- 
termined time is 10 seconds when composing the pictures, 
these values are set as initial values and the channel numbers 
Cj to C 6 of the video signal sources are obtained. These 
measures may be specified in advance depending on the 

55 system. In this case, since the number n of equally divided 
parts of the display screen is n-4, the video signals of the 
channel numbers C a through C 4 are read consecutively, and 
the picture composing process is carried out depending on 
each of the embodiments described above. Thereafter, when 

60 the predetermined time of 10 seconds elapses, the value of 
C A is substituted into C A+1 , and the value of C x is substituted 
into Qy. In other words, (C^l, C 2 «2, C 3 =3, C 4 =4, C 5 =5, 
C 6 -6) is updated to (0^2, C 2 =3, C 3 o4, C 4 «5, C 5 =6, C 6 =l). 
In addition, after the next predetermined time of 10 

65 seconds elapses, (Cj-2, C 2 »3, C 3 =4, C 4 =5, C 5 «6, C 6 «l) is 
updated to (Cj-3, C 2 -4, C 3 -5, C 4 -6, C 5 -l, C 6 »2). 
Accordingly, in the top left block of the picture composing 
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memory 6, for example, the video signal of the channel C 1 ted. The input device is used to make an input similarly to 

is first composed, the video signal of the channel C 2 is next the case of the guidance (a) described above according to the 

composed, the video signal of the channel C 3 is next displayed menu, 

composed, and the position of the blocks when composing Guidance (c) 

the 4 pictures are successively switched according to a 5 According to this guidance, all of the pictures are corn- 
predetermined rotation. In other words, even in a case where posed for N video signal sources wn j c h are effective in all 
the number of video signal sources is large and the display cases A remote menu ^ createdj and mis remote 
screen is divided into a small number of parts, it is possible control meau ^ encode(J and transmitted after an overl 
to successively compose and transmit the video signals from ^ { deyice {s used tQ make an { similarf tQ me 

all of the video signal sources. Therefore, by receiving and _ £ 4 . /x , , , r ' 

& j • « •* • ui * u *u 10 case of the guidance (a) described above according to the 

displaying the composed picture, it is possible to observe the .... & v 7 & 

pictures of the video signals from all of the video signal displayed menu. 

sources Next, a description will be given of a guidance (d) which 

In each of the embodiments described above, a remote is added on lhe monitoring end according to a guidance 

control can easily be made because the picture composing dls P la y instruction from the input device, 

apparatus and the encoder arc closely coupled. For example, 15 Guidance (d) 

low speed data (LSD), multi layer protocol (MLP), high According to this guidance, a remote control menu is 

speed data (HSD), no-standard BAS message and the like created, and only the menu screen is encoded and transmit- 

are prescribed for the user data with respect to the H.320 ted. The input device is used to specify a serial number 

data path, and the remote control can be made using such added to the displayed menu. 

user data. 20 FIGS. 36A, 36B and 36C respectively are diagrams for 

FIG. 35 is a diagram for generally explaining the remote explaining a remote control guidance. FIG. 36A shows 

control in a sixth embodiment of the multiple picture com- display contents corresponding to the guidance (b) described 

posing apparatus according to the present invention. In FIG. above. FIG. 36B shows display contents corresponding to 

35, television cameras 71-1 through 71-n, television cameras the guidance (a) described above. In addition, FIG. 36C 

81-1 through 81 -n, stored video reproducing units 72 and 82, 25 shows display contents corresponding to the guidance (c) 

television monitors 73 and 83, input devices 74 and 84 such described above. The display contents shown in FIG. 36C 

as keyboard, infrared key pad, mouse and ten-key, video are obtained by composing the display contents shown in 

signal compressing and transmitting units (CODECs) 75 and FIGS. 36A and 36B. The display contents shown in FIG. 

85, far-site picture composing information storing memories 36A, 36B or 36C can be displayed on the television monitors 

76 and 86, and near-site picture composing information 30 73 and 83. Of course, it is also possible to display only the 

storing memories 77 and 87 which are coupled as shown. contents of the guidance on a liquid crystal display or the 

The video signal compressing and transmitting units 75 like which is independent of the television monitors 73 and 

and 85 have the functions of exchanging commands, 83. 

responses and indications or notifying messages, and com- FIG. 37 is a diagram for explaining a picture switching by 

posing and encoding the video signals from the television 35 the remote control. When a serial number "4" is specified 

cameras 71-1 through 71-n and 81-1 through 81-n according from the near-site and a single video signal source selection 

to the embodiments described above. The picture composing is input on a display screen 91 corresponding to the guidance 

information is managed by the far-site picture composing (c), a corresponding command is transmitted to the far-site, 

information storing memories 76 and 86 and the near-site and a response with respect to this command is transmitted 

picture composing information storing memories 77 and 87. 40 from the far-site to the near-site. Accordingly, a display 

Modified contents in the far-site picture composing infor- screen 92 for monitoring the volcano is displayed. The menu 

mation storing memory 76 are notified to the near-site is also displayed in this case, but the illustration thereof is 

picture composing information storing memory 87 and vice omitted in FIG. 37 for the sake of convenience, 

versa, and modified contents of the near-site picture com- Next, when a serial number "5" is specified on the menu 

posing information storing memory 77 is are notified to the 45 and a 4-picture composing process is selected, a multiple 

far-site picture composing information storing memory 86 picture composing set command is transmitted to the far- 

and vice versa. site, and a response with respect to this multiple picture 

Various guidances may be prepared in order to carry out composing set command is received from the far-site. As a 

a remote control of the picture composing process. For result, a display screen 93 becomes a composed picture of 4 

example, a description will be given of guidances (a), (b) 50 pictures, similarly to the initial display screen 91. 

and (c) from a monitored end which is monitored when a FIG. 38 is a time chart for explaining a picture composing 

remote menu set/cancel command is transmitted to the mode determination sequence. FIG. 38 shows a case where 

monitored end in accordance with a guidance display a picture com posing mode of a monitored end (far-site) is set 

instruction from the input device 74 or 84. by an input from an operator of a monitoring end (near-site). 

Guidance (a) 55 When the operator uses the input device to input a guidance 

According to this guidance, all of the pictures are com- set request, a remote guidance set command is transmitted 

posed for N video signal sources which are effective in all from the monitoring end (near-site) to the monitored end 

cases. Character data of volcano, river, factory, power line (far-site). A response with respect to this remote guidance set 

and the like which arc registered in advance as the moni- command is transmitted from the monitored end (far-site) to 

toring pictures and serial numbers assigned to the monitor- 60 the monitoring end (near-site), and a guidance (main screen) 

ing pictures are displayed in an overlapping manner. One or is turned ON. 

a plurality of displayed serial numbers are specified by the When the operator inputs a guidance set request based on 

input device, and the picture composing process is carried the guidance (main screen), a remote guidance set command 

out by remote control. is transmitted from the monitoring end to the monitored end. 

Guidance (b) 65 A response with respect to this remote guidance set com- 

According to this guidance, a remote control menu is mand and a next guidance (sub screen) are transmitted from 

created, and only the menu screen is encoded and transmit- the monitored end. When the operator makes an input 
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related to a determination which is made based on the 
guidance (sub screen), a multiple picture composing set 
command is transmitted from the monitoring end to the 
monitored end. A response with respect to this multiple 
picture composing set command and specified contents of 5 
pictures to be composed are transmitted from the monitored 
end to the monitoring end. Furthermore, when the operator 
inputs a guidance cancel request, a remote guidance cancel 
command is transmitted from the monitoring end to the 
monitored end. A response with respect to this remote 10 
guidance cancel command and contents of the pictures to be 
composed with the guidance turned OFF are transmitted 
from the monitored end to the monitoring end. 

FIG. 39 is a diagram for explaining a remote control 
protocol. A remote control protocol 100 related to the picture is 
composing process is generally categorized into commands, 
responses and indications or notifying messages, and indi- 
cates the existence of a response rsp, the type or details, and 
the contents with respect to each of the commands, 
responses and indications or notifying messages. An iden- 20 
tifier of the type or details is indicated by a hexadecimal 
value (hex). Identifiers of commands, parameters and the 
like which will be described later and shown with the (hex) 
symbol omitted are also indicated by a hexadecimal value 
(hex). 25 

The commands include a single video signal source 
selection command having an identifier 01, a multiple pic- 
ture composing set command having an identifier 02, a 
multiple picture composing inquiry command having an 
identifier 03, a video input source state request command 30 
having an identifier 04, and a remote menu set/cancel 
command having an identifier 05, and each of these com- 
mands have a corresponding response rsp. 

The responses are responsive to the commands, and 
include a single video signal source selection response 35 
having an identifier 81 with respect to the single video signal 
source selection command, a multiple picture composing set 
response having an identifier 82 with respect to the multiple 
picture composing set command, a multiple picture com- 
posing inquiry response having an identifier 83 with respect 40 
to the multiple picture composing inquiry command, a video 
input source state request response having an identifier 84 
with respect to the video input source state request 
command, and a remote menu set/cancel response having an 
identifier 85 with respect to the remote menu set/cancel 45 
command. Each of these responses have no corresponding 
response rsp. 

In addition, the indications include a video signal abnor- 
mality indication having an identifier fl, and a multiple 
picture composing modification indication having an iden- 50 
tifier £2. Each of these indications have no corresponding 
response rsp, 

FIGS. 40A and 40B respectively are diagrams for explain- 
ing the single video signal source selection command and 
the single video signal source selection response. 55 

A single video signal source selection command 101 
shown in FIG. 40A includes a 1-byte identifier 01, a 1-byte 
parameter length, and parameters. The parameters include 1 
byte for indicating a total number of channels of the video 
signal sources, and 1 byte for indicating a specified channel 60 
number of the video signal source. 

A single video signal source selection response 102 
shown in FIG. 40B includes a 1-byte identifier 81, a 1-byte 
parameter length, and parameters including 1 byte for indi- 
cating the selected channel number. 65 

FIGS. 41Aand 41B respectively are diagrams for explain- 
ing the multiple picture composing set command. 
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As shown in FIGS. 41 and 41B, a multiple picture 
composing set command 103 includes a 1-byte identifier 02, 
a 1-byte parameter length, 1-byte or 2-byte parameters. The 
parameters include the number N of pictures to be 
composed, the number n of parts into which the display 
screen is divided, an interval time, the picture composing 
mode, and a specified priority channel for the picture com- 
posing mode shown in FIG. 41 A. The parameters further 
include an encoding mode, a specified priority channel for 
the encoding mode, coordinate information related to coor- 
dinates 1 to 9, zoom-up coordinate X, zoom-up coordinate 
Y, and zoom -up ratio shown in FIG. 41 B. 

FIGS. 42A, 42B, 42C, 42D and 42E respectively are 
diagrams for explaining the multiple picture composing 
response, the multiple picture composing inquiry command, 
the multiple picture composing inquiry response, the video 
signal source state request command and the video input 
source state request response. 

As shown in FIG. 42A, a multiple picture composing 
response 104 includes a 1-byte identifier 82, a 1-byte param- 
eter length, and 1-byte parameters. This multiple picture 
composing response 104 is the response with respect to the 
multiple picture composing set command 103 shown in 
FIGS. 41A and 41B. 

As shown in FIG. 42B, a multiple picture composing 
inquiry command 105 includes a 1-byte identifier 03, and a 
1-byte parameter length. 

As shown in FIG. 42C, a multiple picture composing 
inquiry response 106 includes a 1-byte identifier 83, a 1-byte 
parameter length, and 21 bytes of 1-byte or 2-byte param- 
eters. This multiple picture composing inquiry response 106 
is the response with respect to the multiple picture compos- 
ing inquiry command 105 shown in FIG. 42B. 

As shown in FIG. 42D, a video input source state request 
command 107 includes a 1-byte identifier 04, and a 1-byte 
parameter length. This video input source state request 
command 107 inquires the state of the video signal source 
such as the television camera. 

As shown in FIG. 42E, a video input source state request 
response 108 includes a 1-byte identifier 84, a 1-byte param- 
eter length, and 1-byte parameters related to video signal 
sources 1 to 9. This video input source state request response 
108 is the response with respect to the video input source 
state request command 107 shown in FIG. 42D, and each 
1-byte parameter indicates whether the corresponding video 
signal source is normal (00) or abnormal (fl). 

FIGS. 43 A, 43B, 43C and 43D respectively are diagrams 
for explaining the remote guidance set/cancel command, the 
remote guidance set/cancel response, the video signal abnor- 
mality indication and the multiple picture composing modi- 
fication indication. 

A remote guidance set/cancel command 109 shown in 
FIG. 43A includes a 1-byte identifier 05, a 1-byte parameter 
length, and 1-byte parameters. This remote guidance set/ 
cancel command 109 corresponds to the remote guidance set 
command shown in FIG. 38. 

A remote guidance set/cancel response 110 shown in FIG. 
43B includes a 1-byte identifier 85, a 1-byte parameter 
length, and a 1-byte parameter. This remote guidance set/ 
cancel response 110 is the response with respect to the 
remote guidance set/cancel command 109 shown in FIG. 
43A, and corresponds to the remote guidance set response 
shown in FIG. 38. 

A video signal abnormality indication 111 shown in FIG. 
43C includes a 1-byte identifier fl, a 1-byte parameter 
length, and a 1-byte parameter. This video signal abnormal- 
ity indication 111 shows the details of the indication shown 
in FIG. 39. 
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A multiple picture composing modification indication 112 
shown in FIG. 43D includes a 1-byte identifier £2, a 1-byte 
parameter length, and 21 bytes of 1-byte or 2-byte param- 
eters. This multiple picture composing modification indica- 
tion 112 shows the details of the indication shown in FIG. 
39. 

By use of the commands, responses and indications 
described above, it is possible to carry out the remote control 
of the multiple picture composing process in the system 
shown in FIG. 3 or 35. Accordingly, in a remote monitoring 
system or the like, it becomes possible to carry out various 
kinds of control with respect to the picture composing 
process, such as the selection and zoom -up of the picture of 
the monitoring object which is to be monitored. 

Of course, it is possible to appropriately combine the 
above described embodiments to suit the needs. In addition, 
the video signal source is not limited to the television 
camera, and other apparatuses such as a recording and 
reproducing apparatus and a digital camera which output a 
video signal corresponding to a video signal of a still 
camera. Moreover, the application of the present invention is 
not limited to the remote monitoring system, and the present 
invention is similarly applicable to various systems which 
require a multiple picture composing process such as a video 
conference system. 

Further, the present invention is not limited to these 
embodiments, but various variations and modifications may 
be made without departing from the scope of the present 
invention. 

What is claimed is: 

1. A multiple picture composing method for composing 
video signals from a plurality of video signal sources which 
operate asynchronously to each other, comprising the steps 
of: 

(a) successively selecting the video signals from the video 
signal sources; 

(b) reading the selected video signals in units of frames 
while maintaining horizontal and vertical synchroniza- 
tion of the selected video signals; 

(c) reducing the video signals in the units of frames in 
correspondence with areas allocated for each of pic- 
tures to be composed in a composed picture; 

(d) writing the reduced video signal in blocks of a picture 
composing memory corresponding to the areas allo- 
cated for each of the pictures to be composed; and 

(e) composing the video signals stored in the picture 
composing memory to obtain a composed video signal 
related to a composed picture and subjecting the com- 
posed video signal to at least one of an encoding 
process and a transmission process. 

2. The multiple picture composing method as claimed in 
claim 1, wherein said step (e) carries out at least one of the 
encoding process and the transmission process every time a 
video signal in the units of frames and to be composed is 
read from one of the plurality of video signal sources. 

3. The multiple picture composing method as claimed in 
claim 1, wherein said step (b) successively reads the video 
signals in the units of frames from the plurality of video 
signal source and said step (d) writes the reduced video 
signal in the picture composing memory during an encoding 
process and/or a transmission process, and said step (e) 
carries out at least one of the encoding process and the 
transmission process with respect to the video signal stored 
in the picture composing memory upon completion of the 
encoding process and/or the transmission process. 

4. The multiple picture composing method as claimed in 
claim 1, which further comprises the steps of: 
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(f) reading a video signal of a weighted channel a number 
of times greater than video signals of other channels 
and obtaining the composed video signal by writing the 
video signals of the weighted channel and the other 
5 channels in the picture composing memory by said step 
(e). 

5. The multiple picture composing method as claimed in 
claim 1, wherein said step (a) judges existences of the video 
signals from the plurality of video signal sources and selects 

3Q only a video signal source from which a video signal exists, 
and said step (d) writes only the reduced video signal from 
the video signal source selected by said step (a). 

6. The multiple picture composing method as claimed in 
claim 1, which further comprises the steps of: 

(f) writing a fixed pattern in a block of the picture 
15 composing memory corresponding to an invalid area on 

the composed picture, 
said step (e) composing the video signals and the fixed 

pattern stored in the picture composing memory. 

7. The multiple picture composing method as claimed in 
20 claim 1, which further comprises the steps of: 

(f) writing a received and decoded data in a block of the 
picture composing memory corresponding to an invalid 
area on the composed picture, 
25 said step (e) composing the video signals and the received 
and decoded data stored in the picture composing 
memory. 

8. The multiple picture composing method as claimed in 
claim 1, which further comprises the steps of: 

30 (f) writing an enlarged video signal of a weighted channel 
in a block of the picture composing memory corre- 
sponding to an invalid area on the composed picture, 
said step (e) composing the video signals and the enlarged 
video signal stored in the picture composing memory. 
35 9. The multiple picture composing method as claimed in 
claim 1, which further comprises the steps of: 

(f) writing a video signal of a weighted channel in a block 
of the picture composing memory corresponding to an 
invalid area on the composed picture so as to make a 
40 zoom-up display of the video signal of the weighted 
channel. 

10. The multiple picture composing method as claimed in 
claim 1, wherein said step (e) carries out the encoding 
process by setting a quantization value with respect to a 

45 video signal of a weighted channel to a small value with 
priority. 

11. The multiple picture composing method as claimed in 
claim 1, wherein said step (e) carries out an intra-frame 
encoding process with respect to a video signal of a 

50 weighted channel, and carries out an inter-frame prediction 
coding process with respect to video signals of other chan- 
nels. 

12. The multiple picture composing method as claimed in 
claim 1, which further comprises the steps of: 

55 (0 disabling an in-loop filter when carrying out an encod- 
ing process with respect to a video signal of a weighted 
channel, and enabling the in-loop filter when carrying 
out an encoding process with respect to video signals of 
other channels. 

60 13. The multiple, picture composing method as claimed in 
claim 1, which further comprises the steps of: 

(f) setting a motion estimation search range with respect 
to a video signal of a weighted channel to a first range, 
and discontinuing the motion estimation search range 
65 or setting the motion estimation search range with 
respect to video signals of other channels to a second 
range smaller than the first range. 
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14. The multiple picture composing method as claimed in monitoring unit reads the video signals from the video 
claim 10, which further comprises the steps of: signal sources in the units of frames. 

(g) selecting a channel which is to be weighted based on 21 - ^ multiple picture composing apparatus as claimed 

resolution information of the plurality of video signal ID cla ' m ^' ^ rci ° s f id ,P icture composing controller 

sources and frame rate information. s controb said video signal selector so that a video signal of 

15. The multiple picture composing method as claimed in a Righted channel ,s read by said mput signal monitoring 
i • ia u* t c *u • *u . c unit a number of times greater than a number of times video 

claim 10, which further comprises the steps of: sigflals of ^ ^ £ m ^ 

(g) obtaining moving quantity information corresponding 22. The multiple picture composing apparatus as claimed 

to the video signals from the plurality of video signal ^ i n claim 19, which further comprises: 

sources, and weighting a channel having a moving video input judgement means for judging whether or not 

quantity greater than a predetermined value based on the video signals from the plurality of video signal 

the moving quantity information. sources exist, 

16. The multiple picture composing method as claimed in said picture composing controller controls said video signal 
claim 1, wherein said step (d) writes reduced video signals selector to read only a video signal from a video signal 
from the plurality of video signal sources in the units of 15 source from which said input judging means judges an 
frames, further comprising the steps of: existence of the video signal and controls positions of blocks 

(f) enabling an in-loop filter when said step (c) carries out of l . he video si § nals t0 be composed in said picture com- 

the encoding process with respect to the video signals ^ I mem0 7.* , . , - 

immediately after said step (d) writes the video signals, „ . a - J^f**)* picture composing apparatus as claimed 

j U1 . .« . , A u -j * / \ 20 in claim 19, wherein said encoding controller controls at 

and disab mg the in-loop filter when said step (e) ^ oQe oJ ? a tization value with re t t0 a video 

carnes out the encoding process with respect to the &i , corr 4 di t0 a wei hted channel [ n said encoder> 

video signal of a previous frame. switching of an intrtframe encoding process and an inter- 

17. The multiple picture composing method as claimed in frame predictioil coding process in ^ encoderj and 
claim 1, wherein said step (a) selects a number of channels 25 enabled and disabled states of an in-loop filter in the 
of video signal sources depending on a number of pictures encoder. 

to be composed when the number of pictures to be composed 24. The multiple picture composing apparatus as claimed 

is greater than a number of video signal sources, and said in claim 19, which further comprises: 

step (b) switches a channel of the video signal which is read special data storing memory storing a fixed pattern which 

for every predetermined time. 30 is to be stored in a block of said picture composing 

18. The multiple picture composing method as claimed in memory corresponding to an invalid area on the com- 
claim 1, which further comprises the steps of: posed picture. 

(f) transmitting, together with a layout of pictures to be 25 * ^ multiple picture composing apparatus as claimed 

composed, a guidance which includes specifying a in claim 19, which further comprises: 

picture composing process, 35 a decoder decoding received data; and 

said step (e) carrying out the picture composing process a decoding memory storing received data decoded by said 

specified according to the guidance. decoder and to be stored in a block of said picture 

19. A multiple picture composing apparatus for compos- composing memory corresponding to an invalid area on 
ing video signals from a plurality of video signal sources the composed picture. 

which operate asynchronously to each other, comprising: 40 26 * ^ multl P Ie P lcture composing apparatus as claimed 

a video signal selector selecting the video signals from the in claim , 24 > W * GT ™ , said P^re composing controller 

video signal sources* carnes out a control to temporanly store a video signal of a 

. , . weighted channel in said special data storing memory, and 

an input signal monitoring unit reading the video signals tQ store the videQ si al stored h sM {&} data stori 

selected by said video signal selector while maintaining m int0 said icture composing memory so as l0 make 

horizontal and vertical synchronization of the selected ^ a ZQ dis { of tfae videQ si al st0fed m said {&{ 

video signals; data storing memory. 

a horizontal direction reducer reducing the video signals 27. The multiple picture composing method as claimed in 

synchronized by said input signal monitoring unit in c i a i m u, which further comprises the steps of: 

units of frames in correspondence with areas allocated (g) sdectmg a charmel which ^ t0 be weighted based on 

for each of pictures to be composed m a composed resolution information of the plurality of video signal 

picture, sources and frame rate information. 

a picture composing memory storing the video signals 28. The multiple picture composing method as claimed in 

reduced by said horizontal direction reducer in units of daim 12, which further comprises the steps of: 

^ nes; 55 (g) selecting a channel which is to be weighted based on 

a picture composing controller controlling reading of the resolution information of the plurality of video signal 

video signals by said input signal monitoring unit via sources and frame rate information, 

said video signal selector and a picture composing 29. The multiple picture composing method as claimed in 

process with respect to the video signals stored in said claim 13, which further comprises the steps of: 

picture composing memory; and 60 ( g ) selecting a channel which is to be weighted based on 

an encoder carrying out the picture composing process resolution information of the plurality of video signal 

with respect to the video signals stored in said picture sources and frame rate information, 

composing memory. 30. The multiple picture composing method as claimed in 

20. The multiple picture composing apparatus as claimed claim U, which further comprises the steps of: 

in claim 19, which further comprises: 65 ( g ) obtaining moving quantity information corresponding 

an encoding controller controlling said encoder to carry to the video signals from the plurality of video signal 

out the encoding process every time said input signal sources, and weighting a channel having a moving 
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quantity greater than a predetermined value based on 
the moving quantity information. 

31. The multiple picture composing method as claimed in 
claim 12, which further comprises the steps of: 

(g) obtaining moving quantity information corresponding 
to the video signals from the plurality of video signal 
sources, and weighting a channel having a moving 
quantity greater than a predetermined value based on 
the moving quantity information. 
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32. The multiple picture composing method as claimed in 
claim 13, which further comprises the steps of: 

(g) obtaining moving quantity information corresponding 
to the video signals from the plurality of video signal 
sources, and weighting a channel having a moving 
quantity greater than a predetermined value based on 
the moving quantity information. 

* * * * * 



06/30/2004, EAST version: 1.4.1 



